{"id":3975,"date":"2013-08-21T07:00:20","date_gmt":"2013-08-21T15:00:20","guid":{"rendered":"http:\/\/www.formtek.com\/blog\/?p=3975"},"modified":"2013-08-21T07:00:20","modified_gmt":"2013-08-21T15:00:20","slug":"big-data-hadoop-2-0s-yarn-framework-enables-real-time-data-analysis","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/big-data-hadoop-2-0s-yarn-framework-enables-real-time-data-analysis\/","title":{"rendered":"Big Data: Hadoop 2.0's YARN Framework enables Real-Time Data Analysis"},"content":{"rendered":"

Hadoop <\/a>was originally developed primarily as a web crawler for batch indexing the content of web sites. \u00a0Big Data analytics applications have adopted Hadoop, but because of its batch nature of the MapReduce algorithm, it isn’t good at interactively querying and analyzing real-time data streams.<\/p>\n

Hadoop 2.0 attempts to improve and expand on the types of applications to which it can be applied. \u00a0The new framework in\u00a0Hadoop 2.0 is called YARN <\/a>(acronym for \u201cYet Another Resource Negotiator\u201d —\u00a0or MapReduce 2.0). \u00a0It’s more generic than MapReduce and is capable of handling live streams of data and interactive queries.<\/p>\n

Arun Murthy<\/a>, founder of Hortonworks, said that<\/a> “the power of YARN to enable applications to run \u2018in\u2019 Hadoop, instead of \u2018on\u2019 Hadoop, is the key to leveraging all other common services of the next-generation data platform, from security to data lifecycle management… \u00a0You can now have both the batch MapReduce jobs and interactive SQL queries running right next to each other in YARN.”<\/p>\n

Shaun Connolly<\/a>, Hortonworks vice president of corporate strategy for Hortonworks, said that<\/a> “YARN creates a cluster that is aware of all the different types of workloads and resource needs, so they can all cohabitate. You don’t get one workload dominating or taking over all the resources of the cluster.”<\/p>\n

YARN extracts the resource management capability that were previously was embedded in MapReduce and re-packages it in a way so that it can be used by new engines. \u00a0With the new YARN framework, multiple applications can run simultaneously in Hadoop, all going through YARN as the common resource manager.<\/p>\n

The new Hadoop 2.0 YARN framework <\/a>enables:<\/p>\n

\n