Big Data is expected to be a bit of a big bang in terms of it’s explosive growth over the next couple of years. IDC has predicted that the Big Data market will grow from $3.2 billion in 2010 to $16.9 billion in 2015. Peter Sondergaard, senior vice president at Gartner and global head of Research, said that “By 2015, 4.4 million IT jobs globally will be created to support big data, generating 1.9 million IT jobs in the United States.”
It’s interesting to compare the uptake of Big Data in the government and private sectors. Using a definition of Big Data as hardware, software and services that are used for advanced analytics programs, data visualization and data management, research organization Deltek Inc.
estimates that the federal government is spending $5 billion annually now on Big Data. Deltek expects spending by the federal goverment to increase to $7.2 billion by 2017. The report notes that “driving this spending surge is a critical need to turn big data into knowledge so it can be applied for better decision making about countless public safety matters, including thwarting terrorism and developing cures for communicable diseases.”
Just as in the private sector, the data which is being collected by governments is increasing substantially. A survey by research group MeriTalk
found that 87 percent of federal workers say that the volume of data that they’re collecting is increasing, and 96 percent expect that their data volumes will continue to rise over the next two years by an average of 64 percent. Mark Weber
, president of U.S. Public Sector for NetApp, commented to CIO
that ”Government has a gold mine of data at its fingertips. The key is turning that data into high-quality information that can increase efficiencies and inform decisions. Agencies need to look at Big Data solutions that can help them efficiently process, analyze, manage and access data, enabling them to more effectively execute their missions.”
Despite increased spending on Big Data, how well are goverments really doing? An article by Franz Aman
on Forbes, Chief Marketing Officer at SGI, surprisingly suggests that the government may actually be quite a bit ahead of how Big Data is being used in the private sector. Aman comments that a survey of C-level management from top companies finds that 55 percent see no value in Big Data for their organizations. Aman contrasts that with work being done by the federal goverment, pointing to the Army’s use of real-time analysis of intelligence information in Afghanistan using a Hadoop cluster, and the use by the psotal service of Big Data to detect fraudulent use of postage.