The most popular and comprehensive Open Source ECM platform
Let us know more
Data Quality: Poor Quality Impacts AI and Analytics Projects
Data prep is essential to effectively using data-based tools like Data Analytics, Big Data, Machine Learning and AI, as discussed in yesterday’s post. A recent study by O’Reilly backs up the importance of data quality.
The O’Reilly report finds few organizations prepared as they pivot towards becoming more data-driven. In the O’Reilly survey very few teams within businesses are tasked with ensuring data quality. The O’Reilly report suggests that this could signal problems in the near term as data projects grow in importance but data quality fails to keep pace.
As more data sources are added to the incoming analysis stream, problems with inconsistent data become more prevalent. Data problems are expected to get worse. Data governance area where many or most companies are weak include the lack of metadata creation and management, data provenance and data lineage.
Organizations that have started to develop a process around data quality have found that the some of the same technology tools that rely on the data for input, like AI and ML, can also be used to do up-front identification of data quality problems. Nearly half of the businesses who have tried AI-based data quality identification tools found them to be useful.
Will Davis, Trifacta’s senior director of marketing, said that “AI remains an aspirational effort and often times leveraged in very small tests that are easier to control. As organizations begin to use AI for larger initiatives and more AI/ML projects are moved to the cloud, we anticipate more organizations will feel the impact of poor data quality.”
