Big Data's Big Traffic: Have we reached the Point of Disconnect?

IDC/EMC estimate that the total amount of globally stored digital information is somewhere around 1 zetabyte (or one trillion gigabytes) and that stored data is expected to reach 35 zetabytes before 2020.  Forrester Reasearch concurs and estimated that the size of data repositories for business applications will grow 50 percent annually for the foreseeable future.

Technologies like those growing up under the moniker of ‘Big Data’ are likely to only further accelerate the trend towards the growth in data.  Much of the discussion has been focused on how will we cope with being able to store, organize, analyze and use such massive amounts of data.  But one area that isn’t discussed as much is the implications on network traffic resulting from the increased use of large amounts of data.

At a conference held by United Summit and in a recent article by eWeek, the term ‘Big Traffic’ has been showing up.  It’s a description of how the application of massive amounts of data will potentially overwhelm the bandwidth capacity now available on the Internet.  Internet bandwidth will increasingly be driven by technologies like Big Data, the Cloud, machine to machine data exchange, video steaming, and the “Internet of Things“.  All of these technologies are expected to grow dramatically over the coming years and to put an ever greater stress onto the internet infrastructure.

IBM, for example, predicts that internet-based machine-to-machine data communication will grow by 1000 times over the next decade. Cisco predicts that mobile data exchange bandwidth will grow 40 times from now until 2020.  A similar prediction by California San Diego’s Global Information Industry Center (GIIC) found that mobile to mobile data exchange will increase by 18 times over the next four years (from 40 petabytes per month as measured in 2010 to an estimated 451 petabytes per month by 2013).

Michael Kleeman at the University of California at San Diego wrote that “we have reached a point of disconnect between the traditional voice, Internet and broadcast video network architectures and the needs of today’s customers. This disconnect is driven largely by emerging multimedia and multimodal Internet-based traffic and our infrastructures and their underlyingeconomics are struggling to catch up…  Technologically, the price of not keeping up is slower access to websites and greater risk of dropped packets. The typical user will not know what theproblem is, only that their video will not download quickly or their phone calls sound noisy or drop. That, in turn, will affect consumer behavior, which impacts businesses.”

Big Traffic is an important issue that needs to be addressed before it is too late.