Data Virtualization: Abstracting Cloud and On-Premise Data into a Single Logical Access Point

Data virtualization is a method that allows the administration and access to heterogeneous sources of data as if they were a single source.  At the core of data virtualization is the abstraction layer that provides a unified access to disparate back-end data sources,like relational/SQL, Hadoop and NoSQL.  It provides a unified approach to actions like access, query, reporting and predictive analytics.  Data virtualization 50-75 percent efficiencies using techniques of deduplication and consolidation.

Forrester defines data virtualization as “the integration of any data in real-time or near-real time from disparate structured, unstructured, and semi-structured data sources, whether on-premises or cloud into coherent data services that support business transactions, analytics, predictive analytics, and other workloads and patterns.”

The main elements of data virtualization include:

• Abstraction – Abstracts out the need to worry about where and how data is stored and which technologies, APIs and storage structures are used for individual data sets
• Virtualized Data Access – Access to heterogeneous data sources from a single unified logical access point
• Transformation – Ability to convert, clean and transform data into a consumable form
• Data Federation – Combine results sets from across multiple data sources
• Data Delivery – Publish result sets as views and/or data services when requested.

James Kobielus, Big Data evangelist at IBM, wrote that “as the range of your cloudy big data applications grows, you’ll almost certainly have to go further down the virtualization path. The stubborn heterogeneity of hybridized big data clouds will push you in that direction. Within your private clouds, constant big data platform churn will require a virtualization fabric that bridges new approaches with your legacy investments. Churn will stem from your ongoing platform modernization and migration efforts, from your need to incorporate innovative, fit-for-purpose platforms into your cloud, and from vendors’ product-enhancement cycles.”

A recent report by Forrester identifies nine different companies that play in this space. (free download after registration: here)  These include Cisco Systems, Denodo Technologies, IBM, Informatica, Microsoft, Oracle, Red Hat, SAP, and SAS Institute.  The report finds that “the data virtualization market is growing because more Enterprise Architecture pros see data virtualization as a way to address the demand for trusted and secure data in real-time. This market growth is partly due to enterprise architects increasing trust of data virtualization providers to act as strategic partners, advising them on key decisions.”