{"id":6237,"date":"2016-04-14T07:00:16","date_gmt":"2016-04-14T15:00:16","guid":{"rendered":"http:\/\/formtek.com\/blog\/?p=6237"},"modified":"2016-03-12T20:58:56","modified_gmt":"2016-03-13T04:58:56","slug":"big-data-dont-bother-rolling-out-big-data-tools-unless-your-application-really-warrants-it","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/big-data-dont-bother-rolling-out-big-data-tools-unless-your-application-really-warrants-it\/","title":{"rendered":"Big Data: Don’t Bother Rolling Out Big Data Tools Unless Your Application Really Warrants It"},"content":{"rendered":"

When to use Hadoop? \u00a0It really depends on your application. \u00a0And, as we will see, in some cases it doesn’t take big data sets for it to make sense to use Big Data.<\/p>\n

Many, and probably most, applications don’t really have a need to use Big Data tools. Some projects have jumped on the Hadoop bandwagon to apply it to project where\u00a0it might not\u00a0bring much advantage. \u00a0For example, a\u00a0report by Microsoft<\/a> found that the data set sizes for most applications and projects are well under 100 GB.<\/p>\n

Vincent de Lagabbe<\/a>, CTO of Kaiko,\u00a0said that<\/a> \u00a0“considering the volume that we are dealing with, Hadoop would be overkill. It was fancy technology for the time and people started using it for everything, but most things you can do without it. From experience, it’s better to try to do without Hadoop – I mean use it when you need it but when you don’t, don’t bother. \u00a0If you’ve got less than a petabyte of data, Hadoop is probably overkill.”<\/p>\n

On the other end of the spectrum,\u00a0in a ZDNet article<\/a> by Toby Wolpe<\/a>,\u00a0the example of the company\u00a023andMe<\/a>,\u00a0a personal genetic-profiling service, is used. \u00a0It’s the example of an application where it makes sense to use Hadoop but where the actual data set size being analyzed is not that Big.<\/p>\n

When processing a genome sample, the amount of data for one person’s DNA is only about 800 MB of data. \u00a0Mike Gualtieri<\/a>,\u00a0principal analyst at\u00a0Forrester, said\u00a0<\/a>“that’s not a lot. Would you call that big data? If I said 800MB is big data, I’d get laughed out of the room. \u00a0But within that, there are four billion pieces of information and lots of patterns. So it’s a big processing challenge, it’s a big compute challenge. You don’t have to have petabytes of data to have a big-data opportunity or issue.”<\/p>\n

\"Digg<\/a><\/div>
\"Reddit<\/a><\/div>
\"Stumble<\/a><\/div>
\"Buzz<\/a><\/div>
\"Vote<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Kick<\/a><\/div>
\"Shout<\/a><\/div>
\"Share<\/a><\/div>
\"Bookmark<\/a><\/div>
\"Post<\/a><\/div>
\"Google<\/a><\/div><\/div>","protected":false},"excerpt":{"rendered":"

When to use Hadoop? \u00a0It really depends on your application. \u00a0And, as we will see, in some cases it doesn’t take big data sets for it to make sense to use Big Data. Many, and probably most, applications don’t really…<\/span><\/p>\n

Read more ›<\/a><\/div>\n

<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-6237","post","type-post","status-publish","format-standard","hentry","category-big-data"],"_links":{"self":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/6237","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/comments?post=6237"}],"version-history":[{"count":0,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/6237\/revisions"}],"wp:attachment":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/media?parent=6237"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/categories?post=6237"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/tags?post=6237"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}