{"id":4743,"date":"2014-07-25T07:00:09","date_gmt":"2014-07-25T15:00:09","guid":{"rendered":"http:\/\/formtek.com\/blog\/?p=4743"},"modified":"2014-06-28T20:38:23","modified_gmt":"2014-06-29T04:38:23","slug":"data-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/data-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce\/","title":{"rendered":"Data Cloud\/Big Data:  Google Introduces DataFlow as Successor to MapReduce"},"content":{"rendered":"<p>Do you feel left behind when it comes to technologies like Hadoop and MapReduce? \u00a0The great thing about the rapid speed that technology is changing and\u00a0obsolescing is that if you miss one trend it&#8217;s not long before it&#8217;s been superseded by something else. \u00a0That lets you leapfrog directly into the newer technology without having wasted time and resources on the older technology. \u00a0Although you&#8217;ve got to jump in sometime!<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2014\/06\/Hadoop.png\"><img loading=\"lazy\" decoding=\"async\" class=\"aligncenter\" alt=\"Hadoop\" src=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2014\/06\/Hadoop-300x95.png\" width=\"300\" height=\"95\" \/><\/a><\/p>\n<p><a title=\"Goolge announces replacements to MapReduce technologies\" href=\"http:\/\/gigaom.com\/2014\/06\/25\/google-adds-a-big-data-service-and-lots-of-monitoring-to-its-cloud\/\" target=\"_blank\">Google announced<\/a> in June that they&#8217;ve long ago dropped MapReduce technologies like Hadoop. \u00a0And in fact they&#8217;re even going to open up their &#8216;better way&#8217; of analyzing Big Data sets to the public. \u00a0It&#8217;s part of the Google Cloud Platform. \u00a0And the components of the new Google technology called <a title=\"Google DataFlow\" href=\"http:\/\/googlecloudplatform.blogspot.com\/2014\/06\/sneak-peek-google-cloud-dataflow-a-cloud-native-data-processing-service.html\" target=\"_blank\">DataFlow <\/a>have cool names like <a title=\"Google Flume\" href=\"http:\/\/pages.cs.wisc.edu\/~akella\/CS838\/F12\/838-CloudPapers\/FlumeJava.pdf\" target=\"_blank\">Flume <\/a>and <a title=\"MillWheel\" href=\"http:\/\/research.google.com\/pubs\/pub41378.html\" target=\"_blank\">MillWheel<\/a>.<\/p>\n<p>The limitation of MapReduce strategies are that they are run as batch jobs. \u00a0To use MapReduce and standard Hadoop, all the data needs to already exist and to have been collected before the job begins.<\/p>\n<p><a title=\"Greg DeMichillie bio\" href=\"https:\/\/www.linkedin.com\/in\/gregdem\" target=\"_blank\">Greg DeMichillie<\/a>, Director of Product Management, <a title=\"Google on MapReduce\" href=\"http:\/\/googlecloudplatform.blogspot.com\/2014\/06\/reimagining-developer-productivity-and-data-analytics-in-the-cloud-news-from-google-io.html\" target=\"_blank\">wrote that<\/a>\u00a0&#8220;a decade ago, Google invented MapReduce to process massive data sets using distributed computing. \u00a0Since then, more devices and information require more capable analytics pipelines\u2014though they are difficult to create and maintain. \u00a0Cloud Dataflow makes it easy for you to get actionable insights from your data while lowering operational costs without the hassles of deploying, maintaining or scaling infrastructure. You can use Cloud Dataflow for use cases like ETL, batch data processing and streaming analytics, and it will automatically optimize, deploy and manage the code and resources required.&#8221;<\/p>\n<p><a title=\"Brian Goldfarb bio\" href=\"https:\/\/www.linkedin.com\/in\/bgold\" target=\"_blank\">Brian Goldfarb<\/a>, Google Cloud Platform head of marketing, <a title=\"Google's DataFlow Tools\" href=\"http:\/\/www.cmo.com.au\/article\/548562\/google_service_analyzes_live_streaming_data\/\" target=\"_blank\">said that<\/a> with Big Data that &#8220;the program models are different. The technologies are different. It requires developers to learn a lot and manage a lot to make it happen. \u00a0It [Google DataFlow] is a fully managed service that lets you create data pipelines for ingesting, transforming and analyzing arbitrary amounts of data in both batch or streaming mode, using the same programming model.&#8221;<\/p>\n<p><a title=\"Urs Holzle bio\" href=\"www.linkedin.com\/pub\/urs-hoelzle\/65\/683\/b31\" target=\"_blank\">Urs H\u00f6lzle<\/a>, senior vice president of technical infrastructure Google, <a title=\"Holze on Google DataFlow\" href=\"http:\/\/www.datacenterknowledge.com\/archives\/2014\/06\/25\/google-dumps-mapreduce-favor-new-hyper-scale-analytics-system\/\" target=\"_blank\">said that<\/a>\u00a0&#8220;Cloud Dataflow is the result of over a decade of experience in analytics. \u00a0It will run faster and scale better than pretty much any other system out there.&#8221;<\/p>\n<p style=\"text-align: center;\"><a href=\"http:\/\/formtek.com\/blog\/wp-content\/uploads\/2014\/06\/Hadoop.png\">\u00a0<\/a><\/p>\n<div class=\"lightsocial_container\"><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/digg.com\/submit?url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F&amp;title=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/digg.png\" alt=\"Digg This\" title=\"Digg This\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.reddit.com\/submit?url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F&amp;title=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/reddit.png\" alt=\"Reddit This\" title=\"Reddit This\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.stumbleupon.com\/submit?url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F&amp;title=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/stumbleupon.png\" alt=\"Stumble Now!\" title=\"Stumble Now!\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/buzz.yahoo.com\/buzz?targetUrl=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F&amp;headline=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/yahoo_buzz.png\" alt=\"Buzz This\" title=\"Buzz This\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.dzone.com\/links\/add.html?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/dzone.png\" alt=\"Vote on DZone\" title=\"Vote on DZone\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.facebook.com\/sharer.php?t=&amp;u=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/facebook.png\" alt=\"Share on Facebook\" title=\"Share on Facebook\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/delicious.com\/save?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/delicious.png\" alt=\"Bookmark this on Delicious\" title=\"Bookmark this on Delicious\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.dotnetkicks.com\/kick\/?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/dotnetkicks.png\" alt=\"Kick It on DotNetKicks.com\" title=\"Kick It on DotNetKicks.com\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/dotnetshoutout.com\/Submit?title=&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/dotnetshoutout.png\" alt=\"Shout it\" title=\"Shout it\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.linkedin.com\/shareArticle?mini=true&amp;url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F&amp;title=&amp;summary=&amp;source=\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/linkedin.png\" alt=\"Share on LinkedIn\" title=\"Share on LinkedIn\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.technorati.com\/faves?add=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/technorati.png\" alt=\"Bookmark this on Technorati\" title=\"Bookmark this on Technorati\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/twitter.com\/home?status=Reading+https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/twitter.png\" alt=\"Post on Twitter\" title=\"Post on Twitter\" \/><\/a><\/div><div class=\"lightsocial_element\"><a class=\"lightsocial_a\" href=\"http:\/\/www.google.com\/buzz\/post?url=https%3A%2F%2Fformtek.com%2Fblog%2Fdata-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce%2F\" target=\"_blank\"><img decoding=\"async\" class=\"lightsocial_img\" src=\"https:\/\/formtek.com\/blog\/wp-content\/plugins\/light-social\/google_buzz.png\" alt=\"Google Buzz (aka. Google Reader)\" title=\"Google Buzz (aka. Google Reader)\" \/><\/a><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>Do you feel left behind when it comes to technologies like Hadoop and MapReduce? \u00a0The great thing about the rapid speed that technology is changing and\u00a0obsolescing is that if you miss one trend it&#8217;s not long before it&#8217;s been superseded<span class=\"ellipsis\">&hellip;<\/span><\/p>\n<div class=\"read-more\"><a href=\"https:\/\/formtek.com\/blog\/data-cloudbig-data-google-introduces-dataflow-as-successor-to-mapreduce\/\">Read more &#8250;<\/a><\/div>\n<p><!-- end of .read-more --><\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-4743","post","type-post","status-publish","format-standard","hentry","category-big-data"],"_links":{"self":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/4743","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/comments?post=4743"}],"version-history":[{"count":0,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/posts\/4743\/revisions"}],"wp:attachment":[{"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/media?parent=4743"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/categories?post=4743"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/formtek.com\/blog\/wp-json\/wp\/v2\/tags?post=4743"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}