The most popular and comprehensive Open Source ECM platform
Let us know more
AI: Dealing with Advanced Intelligence that is Unpredictable and Inscrutable
AI algorithms have achieved incredible accuracy in identifying disease and solving difficult problems, but the real problem is that we don’t know exactly how AI does what it does. Many of the most widely used and successful AI techniques are complex and opaque in describing how solutions are determined.
Joel Dudley, healthcare director at Mount Sinai, said that “we can build these models, but we don’t know how they work.” This opaqueness of AI results has led to an issue of trust. For mission-critical applications, how can we be certain that AI can be trusted?
Tommi Jaakkola, professor at MIT, said that “it is a problem that is already relevant, and it’s going to be much more relevant in the future. Whether it’s an investment decision, a medical decision, or maybe a military decision, you don’t want to just rely on a ‘black box’ method.”
AI algorithms are also very narrow in their application and crumble when presented edge cases. Mayank Kejriwal, research assistant at USC, wrote that “AIs are not designed to handle highly unexpected situations in an open world. One reason may be the use of modern reinforcement learning itself, which eventually leads the AI to be optimized for the specific environment in which it was trained. In real life, there are no such guarantees. An AI that is built for real life must be able to adapt to novelty in an open world.”
Will Knight, senior writer for WIRED, wrote for MIT Technology Review that “this raises mind-boggling questions. As the technology advances, we might soon cross some threshold beyond which using AI requires a leap of faith. Sure, we humans can’t always truly explain our thought processes either—but we find ways to intuitively trust and gauge people. Will that also be possible with machines that think and make decisions differently from the way a human would? We’ve never before built machines that operate in ways their creators don’t understand. How well can we expect to communicate—and get along with—intelligent machines that could be unpredictable and inscrutable?”
