Behind the Investment: Dremio – Transforming Data Analytics with Cloud Data Lakes, Self-Service, and Lightning Fast Queries

Ask any CIO what their top priorities for the decade ahead are and you’re bound to hear “become a data-driven company” at or near the top. When you dig deeper and unlock what “data driven” really means you’ll hear things like: faster time to insights, accelerating the pace of innovation, unlocking the value of data, enabling self-service analytics, getting a handle on data privacy, etc.

What is tying many of these goals together is the complexity of turning raw data into actionable insights by everyone in the business. The fact is that the tools of yesterday are not equipped to handle the data landscape of today. Data has exploded; it lives in many disparate systems and underlying formats, and analytics tools that were designed to handle a single relational database are struggling to adapt to this heterogenous world. On top of that, data scientists and business analysts are desperate to reduce their dependency on IT by accessing and querying data on their own, with their specific business objectives in mind.

As it stands, however, data engineers are cobbling together multi-layered analytics stacks that combines ETL with custom scripts and numerous 3rd party tools. These stacks are not only complex and slow to set up, but they are also delicate and costly to maintain. To make matters worse, there’s a shortage of data engineers capable of building these stacks and those that are capable are overworked and cannot keep up with demand. Some would say the solution is to move everything into proprietary cloud data warehouses, but that approach just introduces new challenges while tackling some of the existing and is not always the right choice for all types of data. Cloud data lakes can be the answer to many of the issues, but so far, they have been difficult for data engineers and business users alike to work with in ad-hoc use cases to drive rapid business value.

Enter Dremio, a pioneer in the self-service data analytics space offering the world’s first data lake engine. This engine is heavily optimized for open data lake storage, particularly in the cloud with AWS S3 and Microsoft ADLS, but also on-premise with Hadoop. And since not all data lives in data lakes, Dremio can optionally sit on top of external, siloed data sources, and databases. In this way Dremio enables data scientists and business analysts to run analytics in a high performance, open, and self-service environment. The platform couples an Apache Arrow-based query acceleration engine with a self-service semantic layer, providing fast time to insights, avoiding any sort of vendor lock-in, and enabling data governance and security. This caters to the needs of data engineers, business users, and IT alike.

When we at Insight see a game changing business with powerful software, we dig in deep to understand not only the business fundamentals, but the ins and outs of the technology. During diligence on Dremio, we spoke with several of their customers, and the excitement around the product is widespread, from the Global 2000 CIO to the business analyst to the data engineer. We clearly heard that Dremio democratizes data – freeing up overworked data engineers from complex and inefficient data preparation tasks enabling business analysts to easily and quickly run their own data analytics. Implementing Dremio had been the deciding factor between a miserably failed and a hugely successful data lake project for many enterprises.

Getting to know the technology is one piece of it, but we also spent the time really building a relationship with the executive team. We met the co-founder Tomer Shiran, first back in 2016 when the company was still in stealth. Tomer impressed us right off the bat with his product vision and relevant industry experience (former MapR product executive) and from that point on, Dremio was always at the top of our prospect list. We enjoyed staying close to the team as they built up their business to the point that we became a logical partner for Dremio as they continue to scale. With Billy Bosworth, the former CEO of Datastax, recently joining as the CEO, we couldn’t think of a better person to lead the company to the next stage of growth.