The most popular and comprehensive Open Source ECM platform
Let us know more
Adversarial AI: Profiting from Algorithm Deception
Adversarial AI is the purposeful manipulation of machine learning input data with the intent of causing the algorithm to misinterpret the data, causing it to reach conclusions favorable to the attacker. It is like an optical illusion for the algorithm.
The manipulated data is presented in a way to make it resemble normal input but is designed to make the unstable and make inaccurate predictions.
What benefit could hackers get by doing this kind of manipulation? One example given by Jonathan Shaw in Harvard Magazine might be the manipulation of data fed to an algorithm that interprets medical imaging to identify skin cancer. The algorithm could be tricked into identifying benign tumors as malignant, causing the system to order expensive additional and unnecessary treatments.
David Danks, professor at Carnegie Mellon, writing for IEEE Spectrum said that “adversarial attacks can lead to completely bizarre and ridiculous (from a human perspective) behavior from an AI… [this] opens the door to much different types of deception, with much different results. Without proper understanding of these potential impacts, the world is likely to be a less stable and less safe place. “
