In the last decade, we’ve seen the rise of the Modern Data Stack where cloud-native data warehouses and data lakes are the new 'systems of record', replacing brittle ETL (Extract, Transform, and Load) scripts with simpler, more scalable ELT (Extract, Load, Transform) tools. This enables large scale ’schema on read’ capabilities where transformation of data is within the data warehouse/lake itself. In essence, it is an indictment of classical data warehousing is seeing the rise of the Modern Data Stack beyond what Hadoop started, then continued by Spark, and now exploding with cloud-native data warehouses/lakes.
These structural tailwinds have introduced the need for Modern Orchestration where distributed scheduling, complex workflows, data lineage, and observability are essential capabilites for data engineering success. Today, Apache Airflow as a Python based, open-source orchestration platform is used by hundreds of thousands of data teams with over eight million monthly downloads. Apache Airflow hit a major inflection point after December 17, 2020 with the release of V2 which increased speed, updated the core UI, and supported more robust analytical use cases.
Despite its many incredible virtues, Airflow often remains difficult to use and challenging to configure. Enter Astronomer, the Modern Data Orchestration platform, powered by Apache Airflow. Astronomer is the SaaS experience for Airflow that enables:
- Containerized fragmented Airflow deployments, which enables a single point of enterprise control
- Packaged customer support to troubleshoot issues or provide guidance based on a deep library of best practices and documentation
- Integrations options for Airflow operator deployment on virtual private clouds or Astronomer's public cloud
All of this adds up to demand for Astronomer skyrocketing Modern Data teams - more use cases, faster deployment, great support - while reducing the burden on data engineering hours required to deploy and manage Airflow. Astronomer's offerings will continue to expand, including through the acquisition of Datakin, a key driver for the Series C round. Integrating Datakin's data lineage and observability features with Astronomer allows users to access operational lineage and pipeline management within the Modern Data Orchestration platform.
When I first connected with Ry Walker, Astronomer’s founder, the vision of a modern data stack was already tangible. We’re excited the stars have finally aligned for us to work together. Astronomer’s leadership team, now including Joe Otto and Scott Yara, is world class and highly regarded; Astronomer will continue to actively shape the Airflow ecosystem and remain an integral part of its development, adoption and scale. We are thrilled to be leading Astronomer's $213M Series C with participation by Sutter Hill Ventures, Meritech, Salesforce Ventures, Venrock, and Sierra Ventures.
