{"id":11011,"date":"2022-08-19T07:00:00","date_gmt":"2022-08-19T15:00:00","guid":{"rendered":"https:\/\/formtek.com\/blog\/?p=11011"},"modified":"2021-10-13T16:53:25","modified_gmt":"2021-10-14T00:53:25","slug":"artificial-intelligence-accurate-modeling-isnt-possible-without-data-quality","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/artificial-intelligence-accurate-modeling-isnt-possible-without-data-quality\/","title":{"rendered":"Artificial Intelligence: Accurate Modeling isn’t possible without Data Quality"},"content":{"rendered":"\n

Data quality is one of the most important parameters needed in deriving useful answers from Big Data, Data Analytics, Business Intelligence, and AI algorithms, but it is often considered as ‘operational’ and doesn’t get the attention that it rightfully deserves.<\/p>\n\n\n\n

A study done by PwC<\/a> estimates that high-quality ‘clean’ data can save businesses 33 percent on their data-intensive projects and help those companies boost revenues by one-third.<\/p>\n\n\n\n

Tom Redman<\/a>, president at Data Quality Solutions, said that <\/a>“Data quality is everything. The first thing is that if you\u2019re using existing data to train a model and you don\u2019t do a really good job cleaning it up, you\u2019re going to get a bad model. Even if the model [you construct] is good, if you put bad data into it, you\u2019re just going to get a bad result. If you stack these things up, it\u2019s like a cascade, and the problem will quickly get out of control.\u201d<\/p>\n\n\n\n

Data quality trumps data quantity. Researchers from MIT and Amazon found that<\/a> “traditionally, ML practitioners choose which model to deploy based on test accuracy \u2014 our findings advise caution here… Small increases in the prevalence of originally mislabeled test data can destabilize ML benchmarks, indicating that low-capacity models may actually outperform high-capacity models in noisy real-world applications… This gap increases as the prevalence of originally mislabeled test data increases.”<\/p>\n\n\n\n

Tadhg Nagle<\/a>, lecturer at Cork University Business School, wrote in Harvard Business Review<\/a> that “even if you don\u2019t care about data per se, you still must do your work effectively and efficiently. Bad data is a lens into bad work, and [our research shows] that most data is bad. Unless you have strong evidence to the contrary, managers must conclude that bad data is adversely affecting their work.” Nagle estimates that only 3 percent of data sets that they’ve examined meet ‘basic quality standards’.<\/p>\n

\"Digg<\/a><\/div>
\"Reddit<\/a><\/div>
\"Stumble<\/a><\/div>
\"Buzz<\/a><\/div>
\"Vote<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Kick<\/a><\/div>
\"Shout<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Post<\/a><\/div>
\"Google<\/a><\/div><\/div>","protected":false},"excerpt":{"rendered":"

Data quality is one of the most important parameters needed in deriving useful answers from Big Data, Data Analytics, Business Intelligence, and AI algorithms, but it is often considered as ‘operational’ and doesn’t get the attention that it rightfully deserves.…<\/span><\/p>\n

Read more ›<\/a><\/div>\n

<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[69],"tags":[],"class_list":["post-11011","post","type-post","status-publish","format-standard","hentry","category-artificial-intelligence"],"_links":{"self":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/11011","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/comments?post=11011"}],"version-history":[{"count":1,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/11011\/revisions"}],"predecessor-version":[{"id":11012,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/11011\/revisions\/11012"}],"wp:attachment":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/media?parent=11011"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/categories?post=11011"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/tags?post=11011"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}