Saving Millions With Cloud-Based Data Solution and Hortonworks



Webtrends was originally a analytics and a device management company. So, we took device data logs, and process network information, and see what was happening on your infrastructure through the IT world. Everything is processed through relational databases, stored in these big file stores on large NADs appliances and we really didn’t have the ability to do this cloud-based processing. Webtrends, fast forward it 10 years now, and we’re in this world of Big Data. Webtrends is in this unique ability that we can collect anything. We really can collect from any type of device. Let’s take this data stream and data events set, and process it, and store it, in a what people are calling data lakes, and be able to look at this data set and process it, and analyze it for different customers, in different ways, and explore it in real time.

We’re able to take these technologies, pipe that in real time, look at this data set and process new predictions, behavioral analysis. We can do things that allow us to determine ROI for different actions and behavioral patterns. So, really the use cases that we’re seeing become limitless in a sense that our customers all have different verticals that their within, and they all need to be able to drive different business actions and different businesses objectives and basically having a data platform like Hadoop really gives us the ability to tailor our capabilities and offerings at Webtrends for those different verticals, in different ways, and different capabilities.

Hadoop 2

Hadoop 2 has really been an acceleration point. We had a Hadoop one cluster for a long time. We were doing some basic heat map work along with it in Webtrends for our analytics product, and it really basically provided what everybody classically thought of what Hadoop was, right? It was MapReduce and HDFS. With Hadoop 2, it really was groundbreaking in the sense that we really have a whole series of new projects that are available to us at for a company to be able to accelerate their analytics capabilities. Webtrends, being analytics company, we provide this service for other people, and yet at the same time we need to be able to use leverage technologies that are provided by Hadoop, so things like Hive, Spark, Storm, Kafka, PIG, these are all projects that basically enable that capability.

So once Hadoop 2.0 came out, it really started changing the way we look at how we interact with data. Things that we would have had to build ourselves before, on top of Hadoop one, are now becoming a community based project. As many people have said once it goes on open source, it really provides a stepping stone, a leap frogging capability for technology. One of the things that Webtrends is working right now with Hortonworks on, is the ability to take Spark, and Hive, and Hadoop, and be able to tie them together through TEZ and some of the execution processes that we got going on.

So, we’re really interested in doing these large types of DAG operations, these big jobs, so that we can maybe not necessarily pull everything in memory, but be able to offload that to the YARN cluster or the YARN instances to build, to be able execute these jobs in parallel. Webtrends Explore, which is releasing this year, is providing this interactive ability to explore this data, and it’s all powered by Hadoop, and it’s powered by technologies that we are working with Hortonworks with, and being able to do this in real time. So, there are very few applications that can actually explore that large amount of data in a very real time through a customer experience, or custom user interface.


Please enter your comment!
Please enter your name here