The Data Pendulum Swings to Consolidation

Why was the Fivetran-dbt merger all but inevitable?

Fivetran & dbt Labs announced their merger yesterday. The all-stock deal combines two companies into an entity approaching $600 million in ARR.

The beauty of the modern data stack was the explosion in choice. As the cloud exploded onto the scene, the legacy data warehouse was replaced by a collection of fast-moving platforms. In that era, specialization won. The pendulum is now swinging back towards consolidation.

Why? The answer lies in compute economics & revenue scale asymmetry. The table below shows why.

There are three different categories of software within this subset of the ecosystem:

1. Ingestion takes data from software & moves it into a cloud data warehouse. Snowflake acquired Datavolo, which commercializes the open source product Apache NiFi, calling it Openflow. Databricks acquired Arcion for ingestion through change data capture, calling it LakeFlow Connect. Fivetran focuses exclusively on this layer.

2. Transformation means reformatting the data within the cloud data warehouse. Snowflake launched native dbt Projects on Snowflake. Databricks offers Delta Live Tables, native SQL, & Python, plus supports hosted dbt through Databricks Workflows. dbt Core/Cloud is the leading independent transformation tool.

3. Compute revenue is generated when we ask questions of our data. Snowflake remains one of the leaders in structured data analysis with their cloud data warehouse. Databricks’ compute is their own as well.

Here’s the asymmetry in one number. Compute represents 72% of the overall market ($7.6B of $10.6B). As a result of their massive operations, Snowflake & Databricks exert significant gravity within the ecosystem. They have expanded beyond the compute market to impose their presence & capture marginal revenue within customers, pressuring the competitive ecosystem.

That’s not to say these components are independent. George Fraser analyzed Snowflake workloads in September 2024, finding transformation represents 40-45% of total Snowflake compute, which means even at smaller scales, startups can have significant impact on these behemoth businesses.

The Fivetran-dbt merger is an inevitable evolution of a maturing market. Two unicorns must partner to compete against two decacorns. They solve two of the three customer problems. But not yet compute.

One could surmise this consolidation signals the end of the modern data stack. I view it differently. The MDS has succeeded beyond our expectations. The stakes are higher now. Broad platforms, fast growth, & AI-native architectures define the next phase. Expect more consolidation.