Access and Feeds

Big Data: Hadoop Ranks on Top, but its many Challenges Make it Vulnerable

By Dick Weisinger

Hadoop started as a tool intended to jump-start Yahoo’s search engine.  And over the course of the last ten years, Hadoop has morphed into a general-purpose computing platform, in fact, the go-to platform for all things related to Big Data.

Hadoop, of course,  is not the only option for doing Big Data analysis, but it does seem to be pulling away from the other options.  Chris Preimesberger, for example, predicted in eWeek that “Hadoop will distance itself from MongoDB, Cassandra, Couchbase and the numerous NoSQL options to become the safe choice. In stark contrast to the fractured and niche-oriented nature of the alternatives, Hadoop offers a uniform approach to a broad set of APIs for big data analytics (including MapReduce, query languages and database access, with easy integration of leading analytic and search platforms) along with an expanding ecosystem to deliver a wide range of services.”

But despite the mindshare that Hadoop has garnered in the world of Big Data, all is not well with Hadoop.  Hadoop is a complex and challenging platform, one difficult to use productively.

Paul Barsch on Smart Data Collective, speculated that “The truth is that some companies are having wild success reporting, analyzing, and predicting on terabytes and in some cases petabytes of Big Data. But for every eBay, Google, or Amazon or Razorfish there are thousands of companies stumbling, bumbling and fumbling through the process of Big Data analytics with little to show for it.”

With what success are businesses having in their attempts to get Hadoop and Big Data to work for them?  Today Dimensional Research released the findings of a recent survey sponsored by Rainstor that investigated how businesses are using Hadoop to solve Big Data problems.  Of the businesses surveyed that have attempted to use Hadoop, half haven’t gotten past the planning stage, and only about a quarter have managed to see their Hadoop Big-Data projects through to production.  88 percent of companies using Hadoop cited challenges in getting the platform to do what they envisioned.

The main challenges these businesses named in using Hadoop include:

  • 37 percent – Not Real Time
  • 26 percent – Took Too Long to Bring it to Production
  • 25 percent – Required Manual coding
  • 18 percent – Cost of Training and Services

The report also found that Hadoop is a favorite often primarily because of its cost.  The report found that the chief reason why businesses choose to use Hadoop is because of its low cost for scaling.

Digg This
Reddit This
Stumble Now!
Buzz This
Vote on DZone
Share on Facebook
Bookmark this on Delicious
Kick It on DotNetKicks.com
Shout it
Share on LinkedIn
Bookmark this on Technorati
Post on Twitter
Google Buzz (aka. Google Reader)

Leave a Reply

Your email address will not be published. Required fields are marked *

*