The end-to-end cost-performance of real-time analytics: Snowflake vs. ClickHouse Cloud

TL;DR #

A benchmark for real-time analytics needs to measure the system end-to-end: fresh data in, query-ready data maintained, fast answers out, and the cost of keeping that full path running.

That is what CostBench measures. It captures the cost and latency of continuous ingest, data maintenance, and query execution together, because different systems spend work in different places.

A read-only benchmark on a static dataset can show how fast queries return once the data is already prepared. It does not show the cost of making that data query-ready, or whether queries stay fast while fresh data is continuously arriving and being maintained in parallel.

In this post, we use CostBench to compare Snowflake and ClickHouse Cloud across that full real-time analytics path.

Real-time analytics does not start when a query runs #

Real-time analytics starts when fresh data arrives.

Before a dashboard, user, or AI agent can get a fast answer, the system has already done a lot of work: ingesting new rows, organizing them for pruning, maintaining derived tables, preserving freshness, and keeping read capacity available for low-latency queries.

That full path is what CostBench is designed to measure: not just the final read query, but the full running system that turns fresh data into fast answers.

That was also the core idea behind our earlier benchmark on query-ready data. We measured one important aspect of this path: the cost-performance of continuously ingesting data and keeping raw data physically organized for fast analytical reads.

Snowflake responded to that benchmark with a set of recommendations: use different ingestion methods, use larger warehouses for loading, and use Snowflake's newer real-time features where available.

Some of those recommendations improve a Snowflake setup. Others answer a different question, such as how quickly Snowflake can batch-load data, when the objective is maximum throughput rather than sustained query-readiness. But all of them point back to the same principle we started with:

A real-time analytics system should be benchmarked as a full path, not as isolated ingest, maintenance, or read tests.

That is what this post does. We use CostBench to rerun the comparison as an end-to-end benchmark: continuous ingest, raw-data organization, pre-aggregation freshness, and continuous query serving. We test two Snowflake paths: the broadly available standard-table setup and the newer Interactive Tables setup.

All benchmark code and results are available in the CostBench repository. The stock-quotes dataset requires a separate data license, so the data itself cannot be redistributed.

CostBench measures the complete analytics path #

That is the methodology behind CostBench.

CostBench measures cost-performance across the complete analytics path:

① Fresh data arrives continuously.

② The system writes that data and makes it query-ready.

③ Raw data is kept in an ordered layout, so later queries can skip most of the table instead of scanning everything.

④ The system maintains pre-aggregated data, so the lowest-latency queries read even less at query time.

⑤ Finally, the system has to serve fast answers continuously, while ingest and maintenance keep running in the background.

CostBench is not a bulk-load or backfill benchmark
CostBench simulates a real-time analytics system in which fresh data is continuously generated at the source and must become query-ready as it arrives. It is not a bulk-load or backfill benchmark that asks how quickly a system can load data that already exists.

In this post, we apply that methodology to Snowflake and ClickHouse.

What our first benchmark measured: query-ready raw data #

Our original benchmark measured one important part of the full real-time analytics path: the cost of continuously turning newly written raw data into query-ready raw data.

Below is the original setup, along with Snowflake's main comments that it helps clarify.

① Dataset and ordering key #

We used the ClickBench web analytics dataset, with more than 100 columns and the original multi-column sorting key used by the ClickBench workload.

That choice was deliberate. Many ClickBench queries benefit from this ordering in ClickHouse, so to keep later query performance measurements fair, we used the equivalent clustering key in Snowflake.

Snowflake's comment: use a simpler clustering key, and avoid the artificial arrival pattern created by repeating the original ClickBench dataset to reach 100 billion rows.

That is fair, and we address both in the expanded benchmark below. But they actually answer different questions: how data arrives, and what physical layout the system has to maintain for the workload.

② Continuous ingest at a fixed rate #

We ingested data continuously at roughly 1 million rows per second, or about 1 GB of uncompressed data per second.

This was not a maximum-throughput loading test. The ingest rate was intentionally fixed to simulate a real-time workload where new data arrives continuously, and the goal was to use the smallest, lowest-cost Snowflake setup that could sustain that rate while keeping the table query-ready.

Snowflake's comment: use COPY INTO, Snowpipe Streaming, larger warehouses, and Gen2 warehouses.

Those are reasonable choices for other ingest tests. But in this benchmark, the loading mechanism was secondary. The question was not how quickly Snowflake could finish loading 100 billion rows, but what it would cost to keep up with a fixed real-time ingest rate as common in real-time analytics use cases.

③ Query-ready raw data, not pre-aggregation #

The first benchmark focused on the base table: newly written raw data in, query-ready raw data out. It did not measure materialized views.

Snowflake's comment: broader production setups should include more of the data pipeline.

Agreed. That is exactly what CostBench adds next: derived-data maintenance, freshness, and reads alongside ingest and raw-data organization.

④ Reads measured after loading #

After the table reached 100 billion rows, we ran the ClickBench query workload once to validate query performance.

So the first benchmark did not measure continuous serving while ingest, clustering, refresh, or other maintenance work continued in the background.

Snowflake's comment: use Interactive Tables and Interactive Warehouses for high-concurrency, low-latency serving.

Fair enough. Interactive Tables were not part of the first benchmark because that benchmark used the Snowflake path most customers could actually use: standard tables with clustering. Interactive Tables are a newer feature and are currently available only in selected regions.

For the expanded benchmark, we therefore test both paths: the broadly available Snowflake setup using standard tables and materialized views, and an Interactive Tables setup in a supported region.

The expanded benchmark addresses the broader question #

Before Snowflake published its response, we had already started applying the new CostBench methodology to a broader full-path benchmark.

This time, we intentionally started at the easier end of the spectrum: a much simpler dataset and ordering key, a cleaner arrival pattern, and continuous reads while data keeps moving, a setup that removes several of the factors Snowflake objected to in the original ClickBench test.

We will repeat this with heavier datasets later. But first, we wanted to measure the full path in the cleanest possible case.

① Simpler dataset, timestamp-ordered ingest #

We use a real stock market quotes dataset licensed from a data provider, with access to hundreds of billions of rows.

Compared with ClickBench, it has a much simpler shape: 12 columns, a simple two-column sorting key, and a natural timestamp order.

We ingest the data in strict timestamp order at steady rate of 1 million rows per second. At that rate, the system receives 100 billion fresh rows every 28 hours.

② Simple clustering key #

This time, the sorting/clustering key are just two columns: (sym, t) - the stock symbol and the timestamp. This is as natural and simple as it can get.

ClickHouse sorts the raw table by those columns, and Snowflake clusters the raw table by those columns.

③ Pre-aggregations included #

Unlike the original benchmark, this run includes continuously maintained pre-aggregations.

That means we also measure the cost of keeping derived data fresh while new rows keep arriving.

④ Continuous queries on the matching layouts #

This time, queries run continuously while fresh data continues to arrive.

The workload includes two query types: dashboard queries against the pre-aggregated data, and drill-down queries against the raw data with filters matching the raw table's ordering/clustering key.

We keep the systems' caches enabled. In benchmarks with static data, we usually disable query-result caches to avoid measuring memory lookups instead of pure engine performance. Here, the data is constantly changing, so caches are much less useful, and leaving them enabled better reflects real-world usage.

We run each query once per round. That mirrors how these queries are used in practice: a dashboard refresh, a user drill-down, or an exploratory ad-hoc query happens once against the latest state of the data.

That simulates the real-time scenario more directly: ingest, maintenance, freshness, and reads all happen at the same time.

ClickHouse Cloud setup #

For ClickHouse Cloud, we used separate services for ingest and reads, so the write path and query path were isolated from each other.

① Client setup #

The client (benchmark driver) is kept intentionally simple and it performs the the same work for ClickHouse and Snowflake.

It reads Parquet row groups directly from existing Parquet files in binary form, combines them into batches of roughly 1 million rows, and sends those batches to the target system. There is no decoding, no decompression, no encoding, and no compression on the client side.

That matters because Snowflake's response called out client-side cost as a missing factor in the first benchmark. In this setup, the client work is identical for both systems and uses very little CPU and memory, so client-side cost is no longer a meaningful differentiator.

② Ingest service #

Ingest runs on a dedicated ClickHouse Cloud service with 2 nodes, each using 2 CPUs and 8 GiB of memory.

That was enough to ingest 1 million rows-per-second data stream continuously, sort the incoming data, and update the pre-aggregated data, while keeping the total number of active data parts across the tables at around 60 at any given time via continuous background merges.

This efficiency also matters in practice: it lets us run demos like StockHouse cost-effectively, with the same market data used in this benchmark streaming in live and becoming query-ready as it arrives.

③ Sorted raw data and materialized views #

ClickHouse writes the raw data directly to disk in sorted form, using a MergeTree table. This is the table that serves the raw-data drill-down workload.

At the same time, an incremental materialized view maintains the pre-aggregated data in an AggregatingMergeTree table as new rows arrive. This is the table that serves the dashboard workload.

Code: raw MergeTree table, AggregatingMergeTree table, and incremental materialized view.

At any time during the benchmark, the raw MergeTree table remains sorted for fast drill-downs, and the AggregatingMergeTree table stores the pre-aggregated data for the dashboard queries, with no freshness gap to the base table. Both tables stay up-to-date and ready for queries all the time.

④ Read service #

Read queries run on a separate ClickHouse Cloud service with 1 node and 16 CPUs.

We use the same amount of read compute for the Snowflake setups, so read-side capacity is aligned across systems.

Note: It is widely understood that a Snowflake Gen2 Small warehouse on AWS uses 16 AWS Graviton3 cores. ClickHouse Cloud also uses AWS Graviton3 cores in AWS deployments, so the comparison aligns both read paths on 16 cores of the same CPU generation.

⑤ Continuous query workload #

To simulate a continuous read workload, we run two types of queries periodically throughout the benchmark.

Every 10 minutes, we send 4 dashboard queries against the pre-aggregated table.

Every hour, we send 2 ad-hoc drill-down queries against the raw table. These drill-down queries use filters that match the raw table's sort order.

We run the same query schedule on Snowflake, with equivalent physical layouts: the raw data is clustered by the same key, and the aggregated data is pre-aggregated in the same way.

Snowflake setup 1: standard tables and materialized views #

The first Snowflake setup uses the path most Snowflake customers can use today: standard tables, standard materialized views, and Gen2 warehouses.

① Client setup #

The client setup (benchmark driver) is the same as in the ClickHouse setup.

It reads Parquet row groups directly in binary form, combines them into batches of roughly 1 million rows, and sends those batches to Snowflake. This time, the client uses COPY INTO to load the Parquet data into the raw table.

As mentioned earlier, the client does no decoding, no decompression, no encoding, and no compression. The amount of performed client-side work is therefore exactly the same as for ClickHouse.

② Ingest warehouse #

For ingest, we again chose the smallest Snowflake warehouse that could sustain the fixed 1 million rows per second ingest rate.

But this time, following Snowflake's recommendation, we use Gen2 hardware: specifically, a Gen2 X-Small warehouse with 8 CPUs.

The goal is still not maximum load throughput.

The goal is the lowest-cost setup that can sustain with the fixed ingest rate to ​​simulate a real-time workload where new data arrives continuously.

③ Serverless clustering #

The raw data is written into a standard Snowflake table. This is the table that serves the hourly raw-data drill-down workload.

The data ordering is then handled by Snowflake's serverless clustering service in the background, as in the original benchmark. The goal is to keep the raw table physically optimized for the drill-down queries.

Code: raw Snowflake table.

④ Materialized view refresh #

To include pre-aggregation in the end-to-end benchmark, we use a Snowflake materialized view over the raw table. This view is queried by the dashboard workload.

Materialized views are an Enterprise-only Snowflake feature. Refresh work is handled by Snowflake's serverless materialized view refresh service in the background.

Code: Snowflake materialized view.

⑤ Read warehouse #

As in the ClickHouse setup, we use separate warehouses for ingest and reads, so the write path and query path are isolated from each other.

For reads, we use a Gen2 Small warehouse with 16 CPUs.

This matches the read-side CPU count used in ClickHouse.

⑥ Continuous query workload #

We run the same continuous query schedule as in ClickHouse.

Every 10 minutes, we run 4 queries against the pre-aggregated data, simulating dashboard refreshes.

Every hour, we run 2 raw-data drill-down queries against the raw table.

Snowflake setup 2: Interactive Tables #

The second Snowflake setup uses Interactive Tables for both the raw data and the pre-aggregated data.

Interactive Tables are currently available only in selected regions. To test them, we had to create a Snowflake account in a supported region. For that reason, we include both Snowflake paths: the broadly available setup using standard tables and materialized views, and the newer Interactive Tables setup where available.

① Client setup #

The client is the same as in the ClickHouse setup and the first Snowflake setup.

It reads Parquet row groups directly in binary form, combines them into batches of roughly 1 million rows, and uses COPY INTO to load the data into Snowflake. The client does no decoding, no decompression, no encoding, and no compression.

② Ingest warehouse #

As before, ingest uses the smallest Snowflake warehouse that could sustain the fixed 1 million rows per second ingest rate.

We use a Gen2 X-Small warehouse with 8 CPUs.

③ Raw data refresh #

Incoming data first lands in a standard Snowflake table. The raw Interactive Table is then maintained from that source table by a user-managed warehouse, with refreshes triggered as needed to meet the configured target lag. This corresponds to the standard Interactive Tables pattern.

We use a 10-minute target lag for the raw Interactive Table, which serves the hourly drill-down workload.

(Direct ingest into Interactive Tables is possible through Snowflake-managed ingest paths, which we treat separately because it does not cover the pre-aggregated path measured here.)

Code and docs: standard Snowflake table, raw Interactive Table, target lag refresh behavior, and insert-only limitation.

④ Pre-aggregated data refresh #

The pre-aggregated data is also stored in an Interactive Table.

Here, the refresh warehouse regularly transfers the new rows since the previous refresh, aggregates them, and merges the result into the pre-aggregated Interactive Table.

We use a 1-minute target lag, which is the smallest target lag available.
This is the table that serves the dashboard workload, so we configure the freshest pre-aggregation path Snowflake allows.

That lets us compare it with ClickHouse incremental materialized views, which update on the ingest path, and measure what it costs to get as close as possible to that freshness model in Snowflake. Snowflake also offers Interactive Materialized Views; we will test that path separately.

We use a Gen2 X-Large warehouse for refreshes, after a Small, Medium, and Large warehouse could not reliably keep the pre-aggregation refresh within the 1-minute target lag.

Code and docs: pre-aggregated Interactive Table, aggregation query, target lag behavior, Interactive Materialized Views, and refresh-size comparison.

⑤ Interactive read warehouse #

For reads, we use a Small Interactive Warehouse with 16 CPUs.

This matches the read-side CPU count used for ClickHouse and the first Snowflake setup.

Unlike a standard warehouse, the Interactive Warehouse also comes with a large cache. For a Small Interactive Warehouse, that cache is roughly 600 GB, allowing the working set of Interactive Tables to be kept in memory.

Note that Interactive Warehouses have a 1-hour minimum billing duration, and auto-suspend has a minimum setting of 24 hours. In this benchmark, that billing model fits the workload: we are measuring a continuously running real-time analytics service, with ongoing ingest, refresh, and queries.

⑥ Continuous query workload #

We run the same continuous query schedule as before.

Every 10 minutes, we run 4 queries against the pre-aggregated data, simulating dashboard refreshes.

Every hour, we run 2 raw-data drill-down queries against the raw Interactive table.

Results: ClickHouse vs. Snowflake standard tables #

First, we compare ClickHouse with the Snowflake setup most customers can use today: standard tables, standard materialized views, serverless clustering, serverless MV refresh, and Gen2 warehouses.

Performance: latency and freshness under continuous ingest [#](/blog/real-time-analytics-cost-performance-snow