{"id":4254,"date":"2013-10-31T07:00:44","date_gmt":"2013-10-31T15:00:44","guid":{"rendered":"http:\/\/formtek.com\/blog\/?p=4254"},"modified":"2013-11-12T02:22:49","modified_gmt":"2013-11-12T10:22:49","slug":"next-generation-databases-real-time-scale-out-relational-processing","status":"publish","type":"post","link":"https:\/\/formtek.com\/blog\/next-generation-databases-real-time-scale-out-relational-processing\/","title":{"rendered":"Next Generation Databases: Real-Time Scale-Out Relational Processing"},"content":{"rendered":"

As databases grow, the most common way to stay pace with more data, more transactions, and more load is to scale-up the database hardware. Scaling up involves adding more CPU power, more memory and more and faster disks. But as data sizes continue to grow beyond what even single node servers are able to handle, the need then increases for scaling out hardware to run on multiple clustered nodes. And when the need further expands to be able to process large amounts of data in real time, even newer techniques have to be employed.<\/p>\n

Relational databases have broken down when trying to process very large volumes of real-time data. Instead, real-time data processing has been an area where NoSQL solutions are increasingly being applied. But No-SQL databases haven’t yet been able to match the complexity and richness of the types of data queries that are possible with traditional relational databases, and because of that, some companies are working on developing creative ways to accelerate standard relational database processing.<\/p>\n

Clustrix<\/a>, for example, has pioneered techniques that goes beyond clustering. Their patented technique partitions data among server nodes in a cluster and then collects and aggregates the results. The concept of their approach is similar to how the MapReduce algorithm breaks down a problem into smaller data sets, calculates the result sets in parallel, and then aggregates and sorts the results. Compared to standard database clustering, calculations can be performed significantly faster.<\/p>\n

Robin Purohit<\/a>, CEO of Clustrix, said that the techniques that “we\u2019re using are similar to many analytic solutions. But what\u2019s different about our solution is that we\u2019re doing it completely in a relational database. The patent that we got granted basically describes the data distribution, and a method for slicing and reslicing of data, but you always maintain the ACID and relational properties of the database. That\u2019s what differentiates us from the majority of the NoSQL and data analytics tools today that have given up on that in order to get scale.”<\/p>\n

George Gilbert<\/a>, GigaOM Pro analyst, said that \u201cTransactions must be able to drive decisions in real time. In-memory databases typically have the fastest performance but often trade off capacity of a large data set and analytic sophistication. DBMS that aren\u2019t constrained just to memory can achieve the analytic sophistication and performance that still support transaction processing, although less than a data warehouse.”<\/p>\n

Clustrix features include:<\/p>\n