The most popular and comprehensive Open Source ECM platform
Let us know more
Algorithmic Economy: Unintended Consequences from Algorithmic Errors
Algorithms don’t always get it right. The consequences of errors and bad data in algorithms that manage government welfare and assistance programs have become a target of criticism.
Michele Gilman a professor of Law at the University of Baltimore School of Law, wrote that “the dangers for low-income people arise because digital profiles often operate as gatekeepers to affordable credit, jobs, employment, education, insurance, health care, and other life necessities. There are thousands of horror stories of disabled and needy people being denied desperately needed state support due to an algorithmic eligibility determination.”
Regulations are complex, and sometimes there are inaccuracies and omissions when data is entered, but the result can often be that some individuals or groups can be negatively impacted. When mistakes are made, officials often tend to trust the algorithm rather than poor.
One example is the state of Michigan in 2013. The algorithm implemented to automate state unemployment benefits had errors and wrongly identified 34,000 people as fraud suspects, resulting in loss of benefits for all of those individuals. As a consequence, there were bankruptcies and suicides.
Raheel Ahmad, co-founder of explainx.ai, wrote that “as AI systems become a daily part of our lives, it is our responsibility to ensure the systems that we design, build and deploy are trustworthy and unbiased. Without understanding, monitoring and regulating these AI systems, we leave ourselves vulnerable, and no one likes to be vulnerable.”
