Big Data Analytics and IoT: Glassbeam Harnesses Spark

Glassbeam announced today the integration of their Internet of Things (IoT) analytics platform called SCALAR with Apache Spark.

Glassbeam is a five-year old machine data analytics company that has built a cloud-based analytics platform specifically designed for analyzing large amounts of IoT data.  Glassbeam customers come primarily from the storage, wireless, networking, and medical device industries.  Glassbeam analytic results are able to help businesses manage their IoT activities, like helping manufacturers to proactively monitor and predict part failures.  In August 2014, Glassbeam received a second round of funding worth $2 million, bringing their total venture funding received to date to $8.1 million.

Puneet Pandit, Glassbeam Founder and CEO, explained that the reason why they decided to integrate Spark with the Glassbeam SCALAR product was to “provide Glassbeam the capability to be able to handle a lot more data, truly a lot more data, from the machine data perspective, and to provide more predictive and prescriptive analytics for our customer base.”

Apache Spark is an open source data analytics cluster-computing framework built on top of the Hadoop Distributed File System (HDFS) and originally developed at UC Berkeley.  Not even a year and a half has gone by since the Apache Spark project went live, but in that short period, Spark has grabbed considerable developer mindshare.  For example, a recent survey of 3000 Java developers by Typesafe found in their ranking of preferred cloud technologies that Spark ranked third among the top three, just after Amazon EC2 and Apache Hadoop.  Part of the reason for Spark’s success is it’s speed; Spark can run 100x faster than traditional Hadoop MapReduce architectures.

Pandit explained that “there are three core tenants to our value proposition in what we believe is needed for a true IoT analytics play in the market today.  The first one is a highly scalable, truly multi-tenanted, highly-flexible cloud which can work on any device, any data, any format and can process data very quickly and efficiently and then convert it into information.  The second thing is an application that can carve out, based on user authentication schemes, a slice of each of the analytics…  Thirdly, what we are announcing today, is deeper, more advanced analytics using machine learning techniques.”

Prior to the Spark integration, the product design for the Glassbeam SCALAR platform already included Cassandra as it’s core database.  Cassandra had been chosen because of its distributed architecture which allowed Glassbeam to process large amounts of data across many servers, with high availability and no single point of failure.

In describing the integration with Spark, Pandit said that “Spark will sit on each of the Cassandra nodes or distributed compute engines. So it’s not just the power of distributing data for high resiliency or scalability or performance, but Spark will also provide higher scalability in terms of analytics and aggregations.”

Ebrahim Abbasi, senior vice president at Violin Memory, said that the Glassbeam platform provides “significant value add in the area of machine data analytics. With enhanced machine learning solutions, now imagine unlocking the value of years of machine log data combined with other corporate data.”