---
category: database
category_name: Database
release_count: 20
has_more: true
canonical: https://releases.sh/categories/database
---
## What's Changed
* Add PostgresBranches.Resize client by @nickvanw in https://github.com/planetscale/planetscale-go/pull/318
**Full Changelog**: https://github.com/planetscale/planetscale-go/compare/v0.170.0...v0.171.0
## What's Changed
* ci: bump GitHub Actions to Node 24-compatible versions by @nkaradzhov in https://github.com/redis/node-redis/pull/3298
* chore(tests): bump default docker test image to 8.8.0 by @nkaradzhov in https://github.com/redis/node-redis/pull/3297
* fix(client): align RedisClientOptions RESP default with RedisClientType by @aartisonigra in https://github.com/redis/node-redis/pull/3303
* test(client): fix three flaky tests around socket/connect lifecycle by @nkaradzhov in https://github.com/redis/node-redis/pull/3299
* fix: embed sources in emitted sourcemaps by @raashish1601 in https://github.com/redis/node-redis/pull/3274
* Sentinel scan iterator by @harshrai654 in https://github.com/redis/node-redis/pull/3141
* ci: replace brittle Codecov uploader with codecov-action by @nkaradzhov in https://github.com/redis/node-redis/pull/3314
* Fix/issue 3311 by @aartisonigra in https://github.com/redis/node-redis/pull/3313
* fix(sentinel): preserve seed nodes in sentinelRootNodes after topology update (#3237) by @aartisonigra in https://github.com/redis/node-redis/pull/3306
## New Contributors
* @aartisonigra made their first contribution in https://github.com/redis/node-redis/pull/3303
* @harshrai654 made their first contribution in https://github.com/redis/node-redis/pull/3141
**Full Changelog**: https://github.com/redis/node-redis/compare/redis@6.0.0...redis@6.0.1
## Why we rewrote WAL-G for Postgres backups in Rust: Meet WAL-RUS
# Why we rewrote WAL-G for Postgres backups in Rust: Meet WAL-RUS
Postgres backups are one of those pieces of infrastructure that should be boring. They sit in the background, continuously archiving WAL files, uploading backups, and making sure that when something goes wrong, recovery is possible.
At ClickHouse Cloud, this path is critical. WAL archival is what allows us to preserve durability and recoverability for our Postgres services. WAL-G has been a strong and reliable tool for this job. It is mature, battle-tested, and has served the Postgres community well.
But as we pushed Postgres into tighter and more resource-constrained environments, we started hitting a specific problem: memory predictability.
That led us to build [**WAL-RUS**](https://github.com/ClickHouse/wal-rus), an **open-source** Rust-based implementation of Postgres backup and WAL archival tooling, designed for predictable memory-efficiency and WAL-G compatibility.
## The problem
WAL-G is written in Go, a garbage-collected language. While Go makes it easy to build reliable infrastructure software, garbage-collected runtimes make memory usage harder to predict, especially for long-running services like WAL archival.
The challenge isn't just **resident memory** (memory actively being used), but also **virtual memory** (memory reserved from the operating system). Go's runtime manages its own memory pools and can [reserve significantly more virtual memory](https://go.dev/doc/faq#Why_does_my_Go_process_use_so_much_virtual_memory) than the application is actively using. As workloads change, this footprint can fluctuate in ways that are difficult to reason about and tune. The [Go GC guide](https://go.dev/doc/gc-guide) describes this as a characteristic "sawtooth" pattern, where memory usage grows between garbage collection cycles and then falls after collection, making it difficult to predict peak memory consumption and provision resources efficiently.
For operators, that creates a simple but important problem: **how much memory should be reserved for backup infrastructure?**
The answer is usually "more than necessary" to avoid unexpected memory pressure. Memory budgeted for WAL archival is memory that cannot be confidently allocated to Postgres itself for queries, shared buffers, and page cache. [Postgres runs most reliably with overcommit disabled](https://www.postgresql.org/docs/current/kernel-resources.html#LINUX-MEMORY-OVERCOMMIT), making virtual memory a valuable resource modern software often leaves as an afterthought.
WAL-G remains a proven and reliable tool, but as we scaled Postgres into increasingly resource-constrained environments, we wanted a backup system with a more predictable memory profile, delivering the same functionality while consuming fewer resources and making capacity planning simpler.
## The solution: Introducing WAL-RUS
We weren't looking for new functionality. WAL-G is a mature and reliable backup system we're happy to contribute to. Our goal was to preserve core functionality and compatibility while providing a more predictable resource profile.
WAL-RUS is a Rust implementation of Postgres backup and WAL archival tooling built to address the operational challenges we encountered with memory predictability and resource usage.
**1. Predictable Resource Usage:** Unlike garbage-collected runtimes, Rust gives us direct control over [memory allocation and concurrency](https://github.com/ClickHouse/wal-rus/blob/176a430d021bab6016c828bbd1dbe85ac1396cfc/src/main.rs#L22). WAL-RUS uses bounded worker pools and carefully controlled concurrency, making memory consumption easier to reason about and reducing the need to over-provision resources for backup infrastructure.
**2. Built for Continuous WAL Archival:** WAL-RUS prioritizes WAL-G's daemon architecture. Instead of spawning a new process and establishing new connections for every WAL file, it maintains persistent object storage connections that continuously process archival requests in the background.
**3. Optimized for Streaming Workloads:** WAL archival is fundamentally a streaming problem: read WAL files, compress them, and upload to object storage. WAL-RUS minimizes unnecessary buffering and data copies throughout this pipeline, allowing it to perform the same archival work with a smaller and more predictable memory footprint.
**4. WAL-G Compatibility:** WAL-RUS uses the same `WALG_` configuration variables as WAL-G and is continuously tested for interoperability. WAL-G can read archives generated by WAL-RUS, and WAL-RUS can read archives generated by WAL-G, making migration straightforward for existing deployments.
## Benchmarks
To evaluate WAL-RUS, we built a [reproducible benchmark](https://github.com/ClickHouse/wal-rus/tree/main/bench) that compares WAL-RUS, WAL-G, and pgBackRest under a sustained, WAL-heavy PostgreSQL workload. The benchmark continuously generates WAL, archives it to S3, and measures how efficiently each archiver uses memory while keeping up with WAL generation. To ensure a fair comparison, all three tools were configured with **four concurrent archival workers**.
### Memory usage
Memory efficiency was the primary motivation behind WAL-RUS, making memory consumption the first metric we examined.
WAL-G reached nearly **2.8 GB** of peak virtual memory during the benchmark, while WAL-RUS remained below **1 GB**, a reduction of more than **70%**. WAL-RUS also maintained a stable memory profile throughout the run, making its resource requirements easier to reason about in production environments. pgBackRest deserves credit here as well. As a C-based implementation without a garbage-collected runtime, it has tight control over memory allocation.
### WAL archival throughput
Both WAL-RUS and WAL-G consistently maintained minimal backlog throughout the benchmark, demonstrating they could keep up with the workload being generated. pgBackRest accumulated a larger backlog during periods of intense WAL activity, illustrating their architectural tradeoffs between daemon-based and process-based archival throughput.
### CPU utilization
CPU utilization is less important, but good to keep an eye on. Usage is comparable between all three, primarily computing LZ4 compression.
## Summary and conclusion
WAL-RUS was built to solve a practical problem: delivering reliable PostgreSQL backups and WAL archival with a smaller, more predictable resource footprint. By combining Rust's explicit memory management with a daemonized streaming architecture, WAL-RUS achieves archival throughput comparable to WAL-G while significantly reducing memory consumption.
Importantly, WAL-RUS remains fully compatible with existing WAL-G archives and configuration, making adoption straightforward for existing deployments. WAL-RUS introduces support for using Postgres 17's wal summaries for incremental backups, which we're working to [upstream to WAL-G](https://github.com/wal-g/wal-g/pull/2293).
We didn't build WAL-RUS because WAL-G lacked functionality. WAL-G remains a mature and battle-tested project. We built WAL-RUS because we wanted tighter control over resource usage while preserving compatibility with the ecosystem that WAL-G helped establish.
As we continue to develop and harden the project, we plan to make WAL-RUS the default backup and WAL archival mechanism for our managed Postgres offering in ClickHouse Cloud.
The project is open source, and we welcome feedback, testing, and contributions!
### Try Postgres managed by ClickHouse
ClickHouse + Postgres has become the unified data stack for applications that scale. With Managed Postgres now available in ClickHouse Cloud, this stack is a day-1 decision.
[Sign up](https://clickhouse.com/cloud/postgres?loc=blog-cta-1090-try-postgres-managed-by-clickhouse-sign-up&utm_blogctaid=1090)
## Announcing Silk: a silky smooth fiber runtime for ClickHouse
## TL;DR #
*Silk is a stackful-fiber library and scheduler with a NUMA-aware work-stealing loop, io\_uring as the I/O ground truth, and zero heap allocation in the steady-state hot path. We built it for ClickHouse, and the first place we aim to integrate it is in our distributed cache.*
## What are fibers? What is Silk? #
Fibers are a lightweight user-space execution unit, somewhat like threads. Unlike threads, fibers participate in cooperative multitasking instead of the preemptive multitasking that threads use; allowing fibers to yield their work instead of block on it. This particular behavior is best suited for asynchronous I/O, which is becoming more of a bottleneck in distributed systems as CPUs grow faster and clusters grow larger.
Unlike threads, fibers do not have a rich ecosystem of language support, which is why we created Silk. Silk is a C++ library that gives you a cooperative fiber scheduler, backed by a per-CPU scheduler that uses `io_uring` for asynchronous I/O and steals work between cores when local queues run dry. It is exceptional at executing high-concurrency networking I/O (hint hint: ClickHouse) and also at high-currency file I/O (surprise surprise: also ClickHouse).
The name is a homage to Cilk, the 1994 MIT work-stealing scheduler whose name was itself a portmanteau of "silk" plus C. Silk is meant to position itself in that lineage. The fiber-as-silk-thread metaphor is a side benefit.
What made us write a runtime rather than reach for an existing one off the shelf is the combination of properties we needed from it:
1. A fiber that yields in tens of nanoseconds
2. Work stealing that respects CPU topology
3. No heap allocation in the steady state
4. `io_uring` treated as the I/O ground truth rather than as a backend bolted onto an older reactor design.
None of the off-the-shelf options gets all four. So we wrote one that does, and we ship it with the harness, GDB extension, and BPF profiler that proves we aim to depend on it in ClickHouse.
## Why fibers, why these fibers, and why now? #
ClickHouse already has a concurrency model, and it works. It's the right model for the parts of the engine that look like query execution: long-running threads doing real CPU work, where the per-thread overhead is amortized over millions of rows of computation.
Yet, we need silk for the rest of the engine. If you trace a query through ClickHouse Cloud, increasingly the long pole is not "a thread did a lot of computation," it is "ten thousand tiny operations completed in a particular order, and the slowest of them shaped the tail." This takes an aim at increasing the performance of object-storage I/O, distributed cache lookups, replica coordination, HTTP fan-out. All components that are I/O-bound, highly concurrent, and decided at the 99th and 99.9th percentile. They are exactly the workloads where the cost of one in-flight request is supposed to be a stack pointer, not a kernel thread.
The argument for stackful fibers, over OS threads or stackless C++20 coroutines, is essentially this: OS threads are too expensive to use as the primary unit of concurrency in a database engine. A few microseconds per context switch, kilobytes of stack, and a finite number of them before the kernel starts context-switching itself to death. Stackless coroutines are cheap but viral: every function on a suspension path has to be marked `co_await`\-able, and the compiler's heap allocation elision optimization (HALO) reliably stops firing the moment the coroutine handle escapes to a real scheduler queue. Stackful fibers give you cheap suspension without the language footprint: any function can yield and the stack is a normal stack.
The historical objection to stackful fibers, dating back to the Photon paper from Alibaba, is cache aliasing: fibers allocated from a slab can have stacks that map to the same L1 cache lines, producing pathological eviction. The Photon paper measured a 13% scheduler-level cost from this. Silk's response is that the problem is a property of slab-allocated stacks specifically, not of stackful fibers in general. Each fiber's stack is `mmap`'d from a per-fiber pool with guard pages on either side. There is no slab and no aliasing. The 13% cost does not appear in our benchmarks, because the precondition for it does not exist.
What silk delivers, by its own benchmarks against the field, is roughly the following:
- About 3.6 nanoseconds per fiber yield with cross-CPU work stealing
- About 7.6 microseconds for an `io_uring` ping-pong
- 5.9 million file IOPS at a working configuration
- Roughly fifteen times the throughput of `boost::asio` at one connection, and roughly four times at high concurrency
- Per-CPU lock-free stack performance up to 2068x faster than a global lock-free stack at 32 threads, via `rseq`
Want to test these numbers yourself? They come from a benchmark harness in the repository (`./bb`) that runs the exact same workloads through silk and the comparison tool, with controlled CPU pinning, fixed warmup periods, percentile tracking, and JSON output that anyone can re-run and verify. The methodology is the strongest single aspect of how silk presents itself.
## How does Silk work? #
The scheduler runs one OS thread per CPU, pinned. Each scheduler thread owns a per-CPU `ProcessorState` containing a bounded ready queue (a Vyukov MPMC queue with cache-line-aligned producer/consumer slots), an `io_uring` ring for asynchronous I/O and timer expiry, a sleep tree ordered by deadline, and an eventfd that doubles as a wakeup doorbell. Every fiber-bearing operation (submitting an I/O, waking a waiter, scheduling a new fiber) happens on the CPU that originated it whenever possible. When a CPU's ready queue is empty, the scheduler thread wakes via a persistent `IORING_OP_POLL_ADD_MULTI` on the eventfd and runs a service loop that drains its CQ ring, processes expired sleeps, and looks for work to steal.
Work stealing is topology-aware. At startup, silk reads the system's CPU topology from `/sys` and builds a steal-candidate list per CPU, sorted by estimated cost: hyperthread siblings first (about a microsecond), same-socket cores next (about fifty microseconds), and cross-socket cores last (about five hundred microseconds). When a CPU steals, it walks its candidate list in cost order, with random shuffling within each cost tier to avoid hot-spotting. This technique is a concrete realization of a "NUMA-aware" scheduler, it's not just "we have separate queues," it's "we know which CPUs are cheap to steal from and prefer them."
Topology-aware scheduling aside, silk has another important performant property: **the steady-state runtime does no heap allocation.** Fiber stacks come from a pool that is `mmap`'d at init and never freed. `FiberFuture`, `IoFuture`, `SleepFuture`, and `MultipleWaitState` all live on the caller's stack; the `outstandingCount` accounting in `waitForMultiple` exists precisely because the state is on the stack and the function must not return until all in-flight signals have completed. Every container is intrusive: the queue node, suspended-list entry, lock-free-stack hook, and waiter-table hook are *fields inside the `Fiber` object itself*, not separate allocations. A fiber can be enqueued in three different containers simultaneously and the cost is zero additional bytes of heap. The same applies to `SleepFuture`, which carries its own `StackEntry` and `TreeEntry` fields for the cancel queue and the deadline-ordered tree. After init, the hot path does no allocation at all. Not less than other libraries, zero.
The last important performant property we shipped silk with: `boost::asio` allocating per async operation. C++20 stackless coroutines allocate per coroutine frame on the heap unless HALO fires (which usually doesn't happen with a real scheduler). Among production-grade general-purpose async runtimes, the property of zero hot-path allocation belongs almost exclusively to systems engineered for real-time use: DPDK, Seastar, or parts of the Linux kernel itself. Silk is in that category as a deliberate design choice, and the consequence is that it can be deployed in places where allocator behavior is part of the SLA: query execution under memory pressure, kernel-bypass paths, or latency-sensitive hot loops where a `malloc` on the wrong page fault means a missed deadline. All key hotspots of a highly performant distributed database.
## What are some of the design choices? #
**The synchronization primitives are textbook in shape.** The `FiberFuture` packed-state pattern, the `FiberSequencer` flat-combining loop, and the `FiberMutex` lock-and-flag race-handling are each canonical implementations of patterns that go wrong in subtle ways more often than they're done right. Every memory fence has a paired counterparty. Every CAS uses the strictest necessary ordering and no stronger.
**HALO does not fire with schedulers that handle production workloads.** The standard pitch for C++20 stackless coroutines is "zero overhead because of HALO." HALO requires the coroutine handle never to escape to a scheduler queue. Every real scheduler violates that condition, so the "zero overhead" claim holds for synthetic benchmarks where the scheduler is trivial and breaks for real applications where the scheduler bears real load.
**The race-handling for park-then-wake is the key to throughput.** Every primitive that suspends a fiber has the same shape: optimistically attempt the operation; on failure, set a flag indicating waiters exist; suspend the fiber via a callback that runs after the fiber has fully parked; in the callback, register the fiber as a waiter and re-check for missed wakeups. Once you have read it carefully in `FiberFuture`, the futex, the mutex, and the sequencer all read fast.
**The whole synchronization layer is one pattern.** When you can implement six synchronization primitives in 700 lines because they are all variations on "packed state plus flag CAS plus queue or table plus suspend callback that re-checks," you have found the right abstraction. The construction of the library had each primitive deliberately built on top of the previous one. Six primitives, two patterns, one underlying suspend-callback contract.
**The public API is small.** There are a total of eight verbs in `FiberScheduler`: initialize, destroy, run, schedule, yield, suspend, enqueue/release waiters, and the I/O primitives. The header is under 400 lines and reads like API reference documentation. Stackful state is treated as an implementation detail, not as something the user composes around.
**The benchmarks are reproducible.** Every comparison is apples-to-apples and every run is reproducible from one command. The silk-vs-asio comparison is silk against asio's *better* configuration: enabling asio's io\_uring backend made it slower, not faster. The silk-vs-fio comparison is silk against fio's `--ioengine=io_uring`, not against `psync`.
**The operational tooling is as serious as the code.** A working GDB extension that handles both x86\_64 and aarch64, with frame layouts pulled from the Boost.Context assembly source files. A BPF profiler with on-CPU and off-CPU sampling, capability-gated for unprivileged use. A benchmark harness that runs comparisons against the reference tool for each workload (asio for network I/O, fio for file I/O, sockperf for TCP latency, nginx for HTTP). The GDB extension has its own integration test in CTest. We want to ship a library that is useful outside of our own authors.
**This is the cache-aliasing rebuttal of Photon.** The Photon paper has been circulating as the standard "stackful fibers are slow" reference for years. The argument that the 13% scheduler-level miss rate it measured is an artifact of slab-allocated stacks, not of stackful fibers per se, and that mmap-from-pool with guard pages sidesteps it entirely, has not been published in the form silk presents it. The benchmarks back it up: silk's per-yield cost is in the nanoseconds, not the microseconds you would expect from a 13% miss-rate runtime.
## Ok, but it's not perfect, right? #
While we're proud of what we've authored, we can acknowledge limitations and constraints.
First and foremost, Silk is Linux-only. It depends on io\_uring, eventfd, mmap with guard pages, rseq, and the modern Linux capability model. There is no portability layer for macOS, Windows, or older kernels. This is a deliberate scope choice, as the target is server-class Linux, and supporting kqueue or IOCP would double the surface for a use case the team does not have.
Second, the scheduler is a process-wide singleton, accessed via static methods on `FiberScheduler`. There is no way to instantiate two isolated schedulers in the same process. This makes the API ergonomic but rules out testing scenarios and advanced usage like "one scheduler for latency-critical work, one for batch." We deliberately kept multi-scheduling out of the library's current scope; as it would be a breaking API change to add later.
Third, the fiber API requires entry-point parameters to fit in 64 bytes (`FIBER_PARAMETERS_SIZE`). Larger payloads need to be heap-allocated and passed by pointer. This avoids per-fiber allocation churn for the common case but is a real constraint that surfaces at compile time via `static_assert`.
And last but not least, the profiler shipping with the library is, in its current form, a generic on-CPU and off-CPU sampling profiler. It's useful, but not yet aware of fiber identity, though per-fiber attribution is on our roadmap. The architectural foundation is in place: silk knows which fiber is running on each thread (via `threadFiber` TLS), the GDB extension already demonstrates that suspended fiber stacks can be walked from outside, and the BPF profiler is structured for incremental probe additions. What's missing is the BPF program updates to read the TLS, plus probably one or two USDT probes at the suspend/resume boundary.
## Where will ClickHouse utilize this first? #
While we have many places where fibers can increase our performance, the first probable target is our [**distributed cache**](https://clickhouse.com/blog/building-a-distributed-cache-for-s3). It is network-bound and high-fan-out, having a possibility of a single query touching hundreds of cache nodes. It is tail-latency-sensitive in the way that decides query latency. Every cache request maps cleanly to a single fiber: fan in, do io\_uring reads, fan out, return. The I/O is io\_uring-shaped already, and the working set is dominated by short-lived requests rather than long-running query work, so silk's steady-state zero-allocation property is most visible exactly here. We expect the largest visible wins to be in the tail: the 99th and 99.9th percentiles, where OS scheduler jitter and allocator pauses under thousands of concurrent threads are the dominant contributors, and where silk's per-CPU pinning and zero hot-path allocation give the kernel and the allocator nothing to flinch at. We have already seen this shape on internal benchmarks: at ten thousand concurrent S3-style requests, the fiber executor's 99.9th percentile is roughly 65% better than the equivalent thread-pool executor, even when median throughput is identical and MinIO is the bottleneck on both. Distributed cache is where silk runs smoothest first; the rest of the engine is on a separate timeline, and we will write about each integration as it lands.
## Where can I see more? #
Silk is published at [github.com/ClickHouse/silk](http://github.com/ClickHouse/silk). The repository contains the library, the benchmark harness, the GDB extension, the BPF profiler, and four documents worth opening first: [`docs/scheduler.md`](https://github.com/ClickHouse/silk/blob/main/docs/scheduler.md), [`docs/sync.md`](https://github.com/ClickHouse/silk/blob/main/docs/sync.md), [`docs/coroutines.md`](https://github.com/ClickHouse/silk/blob/main/docs/coroutines.md), and [`docs/perf.md`](https://github.com/ClickHouse/silk/blob/main/docs/perf.md). Every benchmark in this post is reproducible from a clean checkout. If you are working on a Linux server-class C++ system and the workload looks like high-concurrency I/O with strict tail-latency requirements (distributed caches, object-storage clients, RPC fabrics, HTTP fan-out), silk is in a state where it would value being kicked at. Read the docs, run the benchmarks, file issues. ClickHouse moves fast because the layers underneath it are precise. Silk is the next layer underneath, and it is the layer we needed.
Lastly, as we weave Silk into ClickHouse, we'll author more pieces about how it has increased performance. Be sure to stay tuned!
## Weekly Postgres reports now include egress metrics
Weekly database reports for Postgres now include egress from the past week. The new metrics make it easier to track outbound data transfer alongside query performance, schema recommendations, traffic control activity, storage, CPU, and memory usage.
You can manage which databases you receive reports for in your [user settings](https://app.planetscale.com/settings/reports), and read more about [database egress](https://planetscale.com/blog/database-egress) on the blog.
## Announcing the Managed ClickStack MCP Server
## Bringing the ClickStack MCP Server to the Cloud #
Last month at Open House, we introduced the ClickStack MCP server, making the same observability investigation capabilities that power ClickStack available to external agents and AI workflows. Initially released as part of open-source ClickStack, it gave users a way to connect tools such as Claude, Cursor, and Codex directly to their observability data through a set of specialized investigation primitives designed specifically for logs, metrics, and traces.
Today, we're bringing those same capabilities to Managed ClickStack in ClickHouse Cloud.
For Managed ClickStack users, this means agents can now access the same specialized observability tools while benefiting from the underlying ClickStack architecture that allows teams to retain more telemetry for longer, store unsampled logs, metrics, and traces at low cost, and avoid the rollups often used elsewhere to manage observability spend.
These capabilities give agents access to significantly more context. They can reason across longer historical windows, identify trends that emerge over time, and investigate using complete telemetry rather than sampled or aggregated views of the underlying system.
The MCP server also benefits from the broader ClickStack Cloud architecture. Agent workloads can run against dedicated compute resources, independent of ingestion and user-facing workloads, while features such as compute-compute separation, notebooks, dashboards, and collaborative investigation workflows remain available. The result is a straightforward way to bring AI agents to production observability without introducing another system to operate.
## Why a specialized observability MCP? #
There is already a generic ClickHouse MCP server available today, and it works well for broad analytical tasks and SQL-driven exploration. But while building AI Notebooks, we repeatedly found that observability workflows behave differently from general BI workloads. Models perform much better when they operate against structured investigative tools rather than generating raw SQL queries repeatedly.
Raw SQL is powerful, but many observability investigations are awkward to express as one-off queries. Tasks like mining recurring log patterns, comparing behavior across time windows, root causing trace outliers, or following an investigation across logs, metrics, and traces require multi-step analysis and domain-specific logic. Leaving all of that to the model means it has to reconstruct the required query patterns and analysis logic from scratch each time, spending context on query mechanics instead of the problem itself.
The ClickStack MCP server gives agents higher-level semantic tools for observability work. Instead of exposing only a raw SQL interface, it provides stable tools for finding trends in patterns of logs, correlating attributes with outliers, inspecting slow traces, and moving through an investigation with repeatable workflows. Under the hood, those tools still execute optimized ClickHouse queries, but the agent interacts with intent-level operations rather than hand-assembling complex analysis every time.
The result of these tools is that our internal benchmarks show that investigations completed with 25% fewer tool calls, with a 2.5x increase in consistency and an almost 20% increase in evaluation scores vs the standard ClickHouse MCP. A large part of that came from giving the model high-leverage semantic investigation tools instead of forcing it to generate every workflow from raw SQL alone.
### Example tool - event patterns #
As an example, consider the event patterns tool.
Event patterns in ClickStack are used to make sense of large volumes of logs or traces by clustering similar events together.
Instead of scrolling through millions of individual log lines, users get a smaller set of meaningful groups that show which errors are recurring, which are new, and which are driving changes in event volume.
In raw SQL, this is difficult to express efficiently. A model would typically need to generate a query that normalizes the event body, often using regular expressions, then group by the resulting pattern and count occurrences over time. On a large dataset, this scales with the amount of data being scanned and can quickly run into high-cardinality outputs, long query times, or unnecessary server pressure. Quotas and query complexity limits can contain the worst cases, but the user still often ends up narrowing the time range or tuning the underlying data model before the question can be answered.
The ClickStack MCP server takes a different path. For event patterns, it can run a small number of lighter queries, including a random sample of matching events and a count of the total result set. The sample is then processed in the MCP layer using the Drain3 algorithm to identify patterns, with the final counts extrapolated and ranked before being returned to the agent.
This gives the model a much better primitive to work with. The query overhead is fixed rather than growing linearly with the full dataset, the raw output has much lower cardinality, and the tool continues to work well even when the underlying event bodies are highly variable.
### Reusing tools #
These same tools are used inside ClickStack AI Notebooks. The model is not manually stitching together large SQL statements for every step of an investigation. Instead, it works against specialized tools that already understand the underlying observability workflows and ClickStack optimizations.
Loading video...
### Retaining flexibility #
At the same time, we do not think structured investigative tools should completely replace direct SQL access.
One of the reasons ClickHouse works so well for agentic workloads and observability is that SQL remains an incredibly powerful exploratory language. Sometimes an incident eventually reaches a point where there is no higher-level abstraction that helps anymore, and you simply need direct access to the underlying data. The structured tools handle many of the repetitive and common investigation paths efficiently, but SQL remains the escape hatch when engineers or agents need to go deeper, test unusual hypotheses, or answer questions the system was never explicitly designed around.
In practice, the workflows end up complementing each other quite naturally: use optimized investigative primitives for the majority of the investigation, then drop into native queries when the situation calls for it.
### Orchestration, not just investigation #
While some engineers are perfectly happy working directly in the terminal or inside an agent harness like Claude Code, investigations eventually need to be shared with other people. SREs need to collaborate, preserve context, and present evidence once they reach a conclusion.
That is why we do not think observability MCP servers should only expose investigative primitives. Real operational workflows also require orchestration primitives for creating dashboards, persisting searches, managing alerts, and sharing findings across teams.
This becomes especially important for local agent workflows. If an agent investigates an incident locally, the resulting evidence needs to be persisted somewhere for sharing and review by the larger team. Copying raw chat output into documents or generating static reports quickly breaks down during real incidents, leading to inconsistencies.
For that reason, the ClickStack MCP server exposes bi-directional management tools directly inside ClickStack itself. Agents can not only investigate incidents, but also create dashboards, persist searches, and validate that the resulting artifacts contain the required evidence and visualizations.
In practice, investigations naturally evolve into persistent operational artifacts rather than disposable chat histories.
## An MCP server with full context and dedicated compute #
As observability becomes increasingly agent-driven, we expect both the number of queries and the amount of data being analyzed to grow significantly. Unlike traditional dashboards, agents are inherently exploratory, iteratively investigating systems, testing hypotheses, and traversing across logs, metrics, and traces.
At the same time, the quality of these investigations depends heavily on context. Agents benefit from access to longer retention windows for historical analysis and trend discovery, as well as unsampled telemetry that allows them to understand the complete picture. Without that context, an investigation can easily end up following a promising path only to discover that the relevant data was sampled away, aggregated, or no longer retained.
Taken together, these trends place new demands on the underlying observability platform. It needs to retain full-fidelity telemetry cost-effectively for long periods of time while supporting significantly higher query volumes as agents continuously explore, investigate, and validate hypotheses.
[Video](https://clickhouse.com/uploads/clickstack_vs_traditional_82948145ce.html)
Managed ClickStack is designed for exactly these workloads. Built on ClickHouse Cloud, it allows teams to retain large volumes of logs, metrics, and traces cost-effectively while avoiding the sampling and rollups often used to control observability costs.
Just as importantly, agent workloads can be isolated from both ingestion and user-facing workloads using dedicated compute resources. This allows teams to scale agent-driven investigations independently while ensuring production dashboards, alerts, and ingestion pipelines remain unaffected.
## Connecting the MCP Server in Managed ClickStack #
Getting started with the Managed ClickStack MCP server is straightforward. ClickStack on ClickHouse Cloud exposes a managed MCP endpoint at `https://mcp.clickhouse.cloud/clickstack` and uses OAuth 2.0 for authentication.
Before connecting, ensure you have a ClickHouse Cloud service with ClickStack enabled. MCP access can be enabled from the Cloud console by selecting **Connect β Connect with MCP** and toggling MCP support on.
Once enabled, navigate to ClickStack for the service, and select **Team Settings β API & Agents**. Here, ClickStack provides pre-configured connection strings for common MCP clients, including Claude Code, Cursor, and Codex CLI.
Once configured, you'll need to complete authentication. For example, assume you've copied an MCP connection string for Claude.
```
claude mcp add clickstack --transport http https://mcp.clickhouse.cloud/clickstack --header "x-service-id: 11e1031f-9a13-4cac-9bc7-d4ec9286ec17"
```
In this case, launch Claude Code and run /mcp, then select clickstack-cloud to complete the OAuth flow.
> One important detail is that the generated connection string contains a ClickHouse Cloud service ID. This determines which ClickHouse service executes the underlying queries. If you want agent workloads to run on dedicated compute, isolated from your other workloads, you can create a [child service](https://clickhouse.com/docs/cloud/reference/warehouses) and launch ClickStack from that service. The generated MCP configuration will then automatically target the appropriate service, allowing agent-driven investigations to scale independently from ingestion and user-facing workloads.
Once connected, the agent can begin interacting directly with ClickStack's observability primitives. For example, you can ask questions like:
*"Show me the services with the highest error rate over the last 24 hours"*
Under the hood, the MCP server routes these requests through the same optimized investigative tools used by AI Notebooks rather than relying entirely on ad hoc SQL generation.
Suppose we investigate elevated latency in a payment service and eventually determine, through Claude, that the root cause is a cache eviction issue.
At that point, we need a way to persist and share the investigation. We could copy the raw Claude output into a document or ask the model to generate a static HTML report, but neither workflow feels particularly natural.
Below, we use the MCP server to generate a dashboard summarizing the investigation and to persist the findings directly in ClickStack, with a validation step confirming that the dashboard presents the required evidence.
Our resulting dashboard provides a persisted artifact summarizing the incident and presenting evidence for any RCA document.
## Conclusion #
We believe observability is becoming increasingly agent-driven, but for agents to be effective, they need more than access to raw data. They need specialized investigation tools, long-term context, full-fidelity telemetry, and the ability to operate at scale. With the Managed ClickStack
## Select a database in the Postgres web console
Postgres branches can host multiple [logical databases](https://planetscale.com/blog/approaches-to-tenancy-in-postgres), and the web console now lets you pick which one to connect to. Use the database selector in the console header to switch between databases on the same branch without leaving the page.
You can also swap databases from the console prompt with `\connect` or `\c`, just like in `psql`.
**[Read more](https://planetscale.com/docs/postgres/web-console)**
## 8.0.1
# Changes
## π Bug Fixes
- Fix Unix socket maintenance notification handling and tests (#4097)
- Fix async cluster node connection release on write errors (#4111)
- Fixed async MultiDBClient with underlying RedisCluster (#4108)
- Fix hiredis readiness checks for high file descriptors (#4115)
- fix(search): parse RESP3 FT.SEARCH responses with bytes-typed keys (#4109)
- Fixing pubsub's listen method to be blocking. (#4119)
- fix(asyncio): release pooled connection when Pipeline.reset() is cancelled (#4123)
- Avoid per-check fd allocation in hiredis _socket_can_read() β use poll() instead of a per-call selector (#4118)
## π§° Maintenance
- Updating PyJWT dependency. (#4100)
- Update CI badge in README.md (#4099)
- Add missing url query argument parser for ssl_min_version (#4047)
- ci: least-privilege permissions on spellcheck (read) and stale-issues (job-level write for actions/stale) (#4080)
- Bumping github-versions actions (#4102)
- Updating lib version + supported Redis versions in README.md + updating the Redis versions in CI test matrix (#4092)
We'd like to thank all the contributors who worked on this release!
@violuke @mokashang @arpitjain099 @coredumperror @elena-kolevska @vladvildanov @petyaslavova
## Release v26.3.15.4-lts
## The end-to-end cost-performance of real-time analytics: Snowflake vs. ClickHouse Cloud
## TL;DR [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#tldr)
A benchmark for real-time analytics needs to measure the system end-to-end: fresh data in, query-ready data maintained, fast answers out, and the cost of keeping that full path running.
That is what CostBench measures. It captures the cost and latency of continuous ingest, data maintenance, and query execution together, because different systems spend work in different places.
A read-only benchmark on a static dataset can show how fast queries return once the data is already prepared. It does not show the cost of making that data query-ready, or whether queries stay fast while fresh data is continuously arriving and being maintained in parallel.
In this post, we use CostBench to compare Snowflake and ClickHouse Cloud across that full real-time analytics path.
## Real-time analytics does not start when a query runs [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#realtime_analytics_does_not_start_when_a_query_runs)
> Real-time analytics starts when fresh data arrives.
Before a dashboard, user, or AI agent can get a fast answer, the system has already done a lot of work: ingesting new rows, organizing them for pruning, maintaining derived tables, preserving freshness, and keeping read capacity available for low-latency queries.
That full path is what [CostBench](https://clickhouse.com/blog/costbench-data-warehouse-cost-performance) is designed to measure: not just the final read query, but the full running system that turns fresh data into fast answers.
That was also the core idea behind our [earlier benchmark on query-ready data](https://clickhouse.com/blog/write-side-cost-performance-snowflake-clickhouse). We measured one important aspect of this path: the cost-performance of continuously ingesting data **and** keeping raw data physically organized for fast analytical reads.
Snowflake [responded](https://www.snowflake.com/en/blog/engineering/efficient-snowflake-ingestion-query-ready/) to that benchmark with a set of recommendations: use different ingestion methods, use larger warehouses for loading, and use Snowflake's newer real-time features where available.
Some of those recommendations improve a Snowflake setup. Others answer a different question, such as how quickly Snowflake can batch-load data, when the objective is maximum throughput rather than sustained query-readiness. But all of them point back to the same principle we started with:
> A real-time analytics system should be benchmarked as a full path, not as isolated ingest, maintenance, or read tests.
That is what this post does. We use CostBench to rerun the comparison as an end-to-end benchmark: continuous ingest, raw-data organization, pre-aggregation freshness, and continuous query serving. We test two Snowflake paths: the broadly available standard-table setup and the newer Interactive Tables setup.
All benchmark code and results are available in the [CostBench repository](https://github.com/ClickHouse/CostBench/tree/main/full-path-realtime/quotes). The stock-quotes dataset requires a [separate data license](https://github.com/ClickHouse/CostBench/tree/main/full-path-realtime/quotes#data-source--licensing), so the data itself cannot be redistributed.
## CostBench measures the complete analytics path [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#costbench_measures_the_complete_analytics_path)
That is the methodology behind CostBench.
CostBench measures cost-performance across the complete analytics path:
β Fresh data arrives continuously.
β‘ The system writes that data and makes it [query-ready](https://clickhouse.com/blog/write-side-cost-performance-snowflake-clickhouse#what-does-query-ready-mean).
β’ Raw data is kept in an ordered layout, so later queries can skip most of the table instead of scanning everything.
β£ The system maintains pre-aggregated data, so the lowest-latency queries read even less at query time.
β€ Finally, the system has to serve fast answers continuously, while ingest and maintenance keep running in the background.
> **CostBench is not a bulk-load or backfill benchmark**
> CostBench simulates a [real-time analytics system](https://clickhouse.com/blog/selecting-a-real-time-analytical-database) in which fresh data is continuously generated at the source and must become query-ready as it arrives. It is not a bulk-load or backfill benchmark that asks how quickly a system can load data that already exists.
In this post, we apply that methodology to Snowflake and ClickHouse.
## What our first benchmark measured: query-ready raw data [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#what_our_first_benchmark_measured_queryready_raw_data)
Our [original benchmark](https://clickhouse.com/blog/write-side-cost-performance-snowflake-clickhouse) measured one important part of the full [real-time analytics](https://clickhouse.com/blog/selecting-a-real-time-analytical-database) path: the cost of continuously turning newly written raw data into query-ready raw data.
Below is the original setup, along with Snowflake's main comments that it helps clarify.
### **β Dataset and ordering key** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#dataset_and_ordering_key)
We used the [ClickBench](https://benchmark.clickhouse.com/) web analytics dataset, with [more than 100 columns and the original multi-column sorting key](https://github.com/ClickHouse/ClickBench/blob/main/clickhouse-cloud/create.sql) used by the ClickBench [workload](https://github.com/ClickHouse/ClickBench/blob/main/clickhouse-cloud/queries.sql).
That choice was deliberate. Many ClickBench queries benefit from this ordering in ClickHouse, so to keep later query performance measurements fair, we used the equivalent clustering key in Snowflake.
> **Snowflake's comment:** use a simpler clustering key, and avoid the artificial arrival pattern created by repeating the original ClickBench dataset to reach 100 billion rows.
That is fair, and we address both in the expanded benchmark below. But they actually answer different questions: how data arrives, and what physical layout the system has to maintain for the workload.
### **β‘ Continuous ingest at a fixed rate** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#continuous_ingest_at_a_fixed_rate)
We ingested data continuously at roughly 1 million rows per second, or about 1 GB of uncompressed data per second.
This was not a maximum-throughput loading test. The ingest rate was intentionally fixed to simulate a real-time workload where new data arrives continuously, and the goal was to use the smallest, lowest-cost Snowflake setup that could sustain that rate while keeping the table query-ready.
> **Snowflake's comment:** use COPY INTO, Snowpipe Streaming, larger warehouses, and Gen2 warehouses.
Those are reasonable choices for other ingest tests. But in this benchmark, the loading mechanism was secondary. The question was not how quickly Snowflake could finish loading 100 billion rows, but what it would cost to keep up with a fixed real-time ingest rate as common in real-time analytics use cases.
### **β’ Query-ready raw data, not pre-aggregation** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#queryready_raw_data_not_preaggregation)
The first benchmark focused on the base table: newly written raw data in, query-ready raw data out. It did not measure materialized views.
> **Snowflake's comment:** broader production setups should include more of the data pipeline.
Agreed. That is exactly what CostBench adds next: derived-data maintenance, freshness, and reads alongside ingest and raw-data organization.
### **β£ Reads measured after loading** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#reads_measured_after_loading)
After the table reached 100 billion rows, we ran the ClickBench query workload once to validate query performance.
So the first benchmark did not measure continuous serving while ingest, clustering, refresh, or other maintenance work continued in the background.
> **Snowflake's comment:** use Interactive Tables and Interactive Warehouses for high-concurrency, low-latency serving.
Fair enough. Interactive Tables were not part of the first benchmark because that benchmark used the Snowflake path most customers could actually use: standard tables with clustering. Interactive Tables are a newer feature and are currently available only in selected regions.
For the expanded benchmark, we therefore test both paths: the broadly available Snowflake setup using standard tables and materialized views, and an Interactive Tables setup in a supported region.
## The expanded benchmark addresses the broader question [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#the_expanded_benchmark_addresses_the_broader_question)
Before Snowflake published its response, we had already started applying the new CostBench methodology to [a broader full-path benchmark](https://github.com/ClickHouse/CostBench/tree/main/full-path-realtime/quotes).
This time, we intentionally started at the easier end of the spectrum: a much simpler dataset and ordering key, a cleaner arrival pattern, and continuous reads while data keeps moving, a setup that removes several of the factors Snowflake objected to in the original ClickBench test.
We will repeat this with heavier datasets later. But first, we wanted to measure the full path in the cleanest possible case.
### **β Simpler dataset, timestamp-ordered ingest** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#simpler_dataset_timestampordered_ingest)
We use [a real stock market quotes dataset](https://github.com/ClickHouse/CostBench/tree/main/full-path-realtime/quotes#dataset) licensed from a [data provider](https://github.com/ClickHouse/CostBench/tree/main/full-path-realtime/quotes#data-source--licensing), with access to hundreds of billions of rows.
Compared with ClickBench, it has a much simpler shape: [12 columns](https://github.com/ClickHouse/CostBench/tree/main/full-path-realtime/quotes#schema-vendor-agnostic), a simple two-column sorting key, and a natural timestamp order.
We ingest the data in strict timestamp order at steady rate of 1 million rows per second. At that rate, the system receives 100 billion fresh rows every 28 hours.
### **β‘ Simple clustering key** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#simple_clustering_key)
This time, the sorting/clustering key are just two columns: `(sym, t)` - the stock symbol and the timestamp. This is as natural and simple as it can get.
ClickHouse [sorts](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/clickhouse-cloud/create.sql#L22) the raw table by those columns, and Snowflake [clusters](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/snowflake/create.sql#L58) the raw table by those columns.
### **β’ Pre-aggregations included** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#preaggregations_included)
Unlike the original benchmark, this run includes continuously maintained pre-aggregations.
That means we also measure the cost of keeping derived data fresh while new rows keep arriving.
### **β£ Continuous queries on the matching layouts** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#continuous_queries_on_the_matching_layouts)
This time, queries run continuously while fresh data continues to arrive.
The workload includes two query types: **dashboard queries against the pre-aggregated data**, and **drill-down queries against the raw data** with filters matching the raw table's ordering/clustering key.
**We keep the systems' caches enabled**. In benchmarks with static data, we usually disable query-result caches to avoid measuring memory lookups instead of pure engine performance. Here, the data is constantly changing, so caches are much less useful, and leaving them enabled better reflects real-world usage.
**We run each query once per round**. That mirrors how these queries are used in practice: a dashboard refresh, a user drill-down, or an exploratory ad-hoc query happens once against the latest state of the data.
That simulates the real-time scenario more directly: ingest, maintenance, freshness, and reads all happen at the same time.
## ClickHouse Cloud setup [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#clickhouse_cloud_setup)
For ClickHouse Cloud, we used separate services for ingest and reads, so the write path and query path were [isolated](https://clickhouse.com/blog/introducing-warehouses-compute-compute-separation-in-clickhouse-cloud) from each other.
### **β Client setup** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#client_setup)
The [client](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/clickhouse-cloud/ingest_parquet_dir.py) (benchmark driver) is kept intentionally simple and it performs the the same work for ClickHouse and Snowflake.
It reads Parquet row groups directly from existing Parquet files in binary form, combines them into batches of roughly 1 million rows, and sends those batches to the target system. There is no decoding, no decompression, no encoding, and no compression on the client side.
That matters because Snowflake's response called out client-side cost as a missing factor in the first benchmark. In this setup, the client work is identical for both systems and uses very little CPU and memory, so client-side cost is no longer a meaningful differentiator.
### **β‘ Ingest service** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#ingest_service)
Ingest runs on a dedicated ClickHouse Cloud service with 2 nodes, each using 2 CPUs and 8 GiB of memory.
That was [enough](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/clickhouse-cloud/results/utilization/writer/README_clickhouse_writer_utilization.md) to ingest 1 million rows-per-second data stream continuously, sort the incoming data, and update the pre-aggregated data, while keeping the total number of active data parts across the tables at around 60 at any given time via [continuous background merges](https://clickhouse.com/docs/merges).
> This efficiency also matters in practice: it lets us run demos like [StockHouse](https://stockhouse.clickhouse.com/) cost-effectively, with the same market data used in this benchmark streaming in **live** and becoming query-ready as it arrives.
### **β’ Sorted raw data and materialized views** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#sorted_raw_data_and_materialized_views)
ClickHouse writes the raw data directly to disk in [sorted form](https://clickhouse.com/blog/write-side-cost-performance-snowflake-clickhouse#clickhouse-ordering-on-the-write-path), using a MergeTree table. This is the table that serves the raw-data drill-down workload.
At the same time, an incremental materialized view maintains the pre-aggregated data in an AggregatingMergeTree table as new rows arrive. This is the table that serves the dashboard workload.
Code: [raw MergeTree table](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/clickhouse-cloud/create.sql#L7), [AggregatingMergeTree table](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/clickhouse-cloud/create.sql#L31), and [incremental materialized view](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/clickhouse-cloud/create.sql#L53).
> At any time during the benchmark, the raw MergeTree table remains sorted for fast drill-downs, and the AggregatingMergeTree table stores the pre-aggregated data for the dashboard queries, with no freshness gap to the base table. Both tables stay up-to-date and [ready for queries](https://clickhouse.com/blog/write-side-cost-performance-snowflake-clickhouse#what-does-query-ready-mean) all the time.
### **β£ Read service** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#read_service)
Read queries run on a [separate](https://clickhouse.com/blog/introducing-warehouses-compute-compute-separation-in-clickhouse-cloud) ClickHouse Cloud service with 1 node and 16 CPUs.
> We use the same amount of read compute for the Snowflake setups, so read-side capacity is aligned across systems.
Note: It is widely [understood](https://select.dev/posts/snowflake-warehouse-sizing) that a Snowflake Gen2 Small warehouse on AWS [uses](https://medium.com/snowflake-engineering/deep-dive-inside-snowflakes-new-gen2-standard-warehouses-powered-by-aws-graviton3-6aacca73ae2d) 16 AWS Graviton3 cores. ClickHouse Cloud also uses AWS Graviton3 cores in AWS deployments, so the comparison aligns both read paths on 16 cores of the same CPU generation.
### **β€ Continuous query workload** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#continuous_query_workload)
To simulate a continuous read workload, we run two types of queries periodically throughout the benchmark.
Every 10 minutes, we send [4 dashboard queries](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/clickhouse-cloud/queries_mv.sql) against the pre-aggregated table.
Every hour, we send [2 ad-hoc drill-down queries](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/clickhouse-cloud/queries_raw.sql) against the raw table. These drill-down queries use filters that match the raw table's sort order.
> We run the same query schedule on Snowflake, with equivalent physical layouts: the raw data is clustered by the same key, and the aggregated data is pre-aggregated in the same way.
## Snowflake setup 1: standard tables and materialized views [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#snowflake_setup_1_standard_tables_and_materialized_views)
The first Snowflake setup uses the path most Snowflake customers can use today: standard tables, standard [materialized views](https://docs.snowflake.com/en/user-guide/views-materialized), and Gen2 warehouses.
### **β Client setup** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#client_setup_2)
> The [client](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/snowflake/ingest.py) setup (benchmark driver) is the same as in the ClickHouse setup.
It reads Parquet row groups directly in binary form, combines them into batches of roughly 1 million rows, and sends those batches to Snowflake. This time, the client uses `COPY INTO` to load the Parquet data into the raw table.
As mentioned earlier, the client does no decoding, no decompression, no encoding, and no compression. The amount of performed client-side work is therefore exactly the same as for ClickHouse.
### **β‘ Ingest warehouse** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#ingest_warehouse)
For ingest, we again chose the smallest Snowflake warehouse that could sustain the fixed 1 million rows per second ingest rate.
But this time, following Snowflake's recommendation, we use Gen2 hardware: specifically, a Gen2 X-Small warehouse with 8 CPUs.
> The goal is still not maximum load throughput.
The goal is the lowest-cost setup that can sustain with the fixed ingest rate to ββsimulate a real-time workload where new data arrives continuously.
### **β’ Serverless clustering** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#serverless_clustering)
The raw data is written into a standard Snowflake table. This is the table that serves the hourly raw-data drill-down workload.
The data ordering is then handled by Snowflake's [serverless clustering service](https://docs.snowflake.com/en/user-guide/tables-auto-reclustering) in the background, as in the original benchmark. The goal is to keep the raw table physically optimized for the drill-down queries.
Code: [raw Snowflake table](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/snowflake/create.sql#L44).
### **β£ Materialized view refresh** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#materialized_view_refresh)
To include pre-aggregation in the end-to-end benchmark, we use a Snowflake materialized view over the raw table. This view is queried by the dashboard workload.
Materialized views are an Enterprise-only Snowflake feature. Refresh work is handled by Snowflake's [serverless materialized view refresh service](https://docs.snowflake.com/en/user-guide/views-materialized#materialized-views-cost) in the background.
Code: [Snowflake materialized view](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/snowflake/create.sql#L79).
### **β€ Read warehouse** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#read_warehouse)
As in the ClickHouse setup, we use separate warehouses for ingest and reads, so the write path and query path are isolated from each other.
For reads, we use a Gen2 Small warehouse with 16 CPUs.
> This matches the read-side CPU count used in ClickHouse.
### **β₯ Continuous query workload** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#continuous_query_workload_2)
We run the same continuous query schedule as in ClickHouse.
Every 10 minutes, we run [4 queries against the pre-aggregated data](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/snowflake/queries_mv.sql), simulating dashboard refreshes.
Every hour, we run [2 raw-data drill-down queries](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/snowflake/queries_raw.sql) against the raw table.
## **Snowflake setup 2: Interactive Tables** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#snowflake_setup_2_interactive_tables)
The second Snowflake setup uses [Interactive Tables](https://docs.snowflake.com/en/user-guide/interactive) for both the raw data and the pre-aggregated data.
Interactive Tables are currently [available only in selected regions](https://docs.snowflake.com/en/user-guide/interactive#label-interactive-region-availability). To test them, we had to create a Snowflake account in a supported region. For that reason, we include both Snowflake paths: the broadly available setup using standard tables and materialized views, and the newer Interactive Tables setup where available.
### **β Client setup** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#client_setup_3)
The [client](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/snowflake/ingest.py) is the same as in the ClickHouse setup and the first Snowflake setup.
It reads Parquet row groups directly in binary form, combines them into batches of roughly 1 million rows, and uses `COPY INTO` to load the data into Snowflake. The client does no decoding, no decompression, no encoding, and no compression.
### **β‘ Ingest warehouse** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#ingest_warehouse_2)
As before, ingest uses the **smallest Snowflake warehouse** that could sustain the fixed 1 million rows per second ingest rate.
We use a Gen2 X-Small warehouse with 8 CPUs.
### **β’ Raw data refresh** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#raw_data_refresh)
Incoming data first lands in a standard Snowflake table. The raw Interactive Table is then maintained from that source table by a user-managed warehouse, with refreshes triggered as needed to meet the configured target lag. This corresponds to the standard Interactive Tables pattern.
> We use a 10-minute target lag for the raw Interactive Table, which serves the hourly drill-down workload.
*(Direct ingest into Interactive Tables is possible through Snowflake-managed ingest paths, which we treat [separately](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#outlook_more_paths_and_heavier_workloads) because it does not cover the pre-aggregated path measured here.)*
Code and docs: [standard Snowflake table](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/snowflake/create.sql#L44), [raw Interactive Table](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/snowflake/ops/setup_interactive.sh#L59), [target lag refresh behavior](https://docs.snowflake.com/en/user-guide/interactive#specifying-auto-refresh-for-an-interactive-table), and [insert-only limitation](https://docs.snowflake.com/en/user-guide/snowpipe-streaming/snowpipe-streaming-table-support#insert-only-operations).
### **β£ Pre-aggregated data refresh** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#preaggregated_data_refresh)
The pre-aggregated data is also stored in an Interactive Table.
Here, the refresh warehouse regularly transfers the new rows since the previous refresh, aggregates them, and merges the result into the pre-aggregated Interactive Table.
> We use a 1-minute target lag, which is the smallest target lag available.
> This is the table that serves the dashboard workload, so we configure the freshest pre-aggregation path Snowflake allows.
That lets us compare it with ClickHouse incremental materialized views, which update on the ingest path, and measure what it costs to get as close as possible to that freshness model in Snowflake. Snowflake also offers Interactive Materialized Views; we [will](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#outlook_more_paths_and_heavier_workloads) test that path separately.
We use a Gen2 X-Large warehouse for refreshes, after a Small, Medium, and Large warehouse could not reliably keep the pre-aggregation refresh within the 1-minute target lag.
Code and docs: [pre-aggregated Interactive Table](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/snowflake/ops/setup_interactive.sh#L50), [aggregation query](https://github.com/ClickHouse/CostBench/blob/6fb94fd628b100db69bc2bf9f8d6f149eec54601/full-path-realtime/quotes/snowflake/ops/setup_interactive.sh#L54), [target lag behavior](https://docs.snowflake.com/en/user-guide/interactive#specifying-auto-refresh-for-an-interactive-table), [Interactive Materialized Views](https://docs.snowflake.com/en/user-guide/interactive#materialized-view-support-for-interactive-tables), and [refresh-size comparison](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#freshness_scheduled_refresh_has_to_keep_up).
### **β€ Interactive read warehouse** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#interactive_read_warehouse)
For reads, we use a Small [Interactive Warehouse](https://docs.snowflake.com/en/user-guide/interactive) with 16 CPUs.
> This matches the read-side CPU count used for ClickHouse and the first Snowflake setup.
Unlike a standard warehouse, the Interactive Warehouse also comes with a [large cache](https://docs.snowflake.com/en/user-guide/interactive#choosing-a-size-for-an-interactive-warehouse). For a Small Interactive Warehouse, that cache is roughly 600 GB, allowing the working set of Interactive Tables to be kept in memory.
Note that Interactive Warehouses have a 1-hour minimum billing duration, and auto-suspend has a minimum setting of 24 hours. In this benchmark, that billing model fits the workload: we are measuring a continuously running real-time analytics service, with ongoing ingest, refresh, and queries.
### **β₯ Continuous query workload** [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#continuous_query_workload_3)
We run the same continuous query schedule as before.
Every 10 minutes, we run [4 queries against the pre-aggregated data](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/snowflake/queries_mv_it.sql), simulating dashboard refreshes.
Every hour, we run [2 raw-data drill-down queries](https://github.com/ClickHouse/CostBench/blob/main/full-path-realtime/quotes/snowflake/queries_raw_it.sql) against the raw Interactive table.
## Results: ClickHouse vs. Snowflake standard tables [#](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#results_clickhouse_vs_snowflake_standard_tables)
First, we compare ClickHouse with the Snowflake [setup](/blog/real-time-analytics-cost-performance-snowflake-vs-clickhouse#snowflake_setup_1_standard_tables_and_materialized_views) most customers can use today: standard tables, standard materialized views, serverless clustering, serverless MV refresh, and Gen2 warehouses.
### Performance: latency and freshness under continuous ingest [#](/blog/real-time-analytics-cost-performance-snow
## What's new in ClickStack - May 2026
## Summary
Welcome to the June edition of What's New in ClickStack.
June focused on making dashboards more useful during investigations. Dashboard table linking introduces clickable actions that take users directly from aggregated views into searches or related dashboards, while scoped filters make it easier to build multi-source dashboards without every filter applying everywhere.
We also expanded dashboard customization. Number tiles now support colors, threshold-based formatting, and automatic text scaling, making it easier to surface important signals at a glance. The service map received a substantial update as well, with latency percentiles, throughput metrics, server-side filtering, and a new focus mode for exploring dependencies.
Beyond dashboards, we added a Browser RUM dashboard template, experimental PromQL support powered by the ClickHouse TimeSeries Engine, and a large batch of new MCP tools.
## OpenHouse Announcements
We also made several major observability announcements at Open House.
[AI Notebooks entered beta for Managed ClickStack users](https://clickhouse.com/blog/observability-mcp-server-ai-notebooks#ai-notebooks-in-beta), providing a persistent workspace for investigations where engineers can combine prompts, queries, charts, reasoning, and findings in a single shareable artifact. Alongside Notebooks, we introduced the [ClickStack MCP server](https://clickhouse.com/blog/observability-mcp-server-ai-notebooks#clickstack-mcp-server), exposing the same observability workflows and investigative primitives used internally by ClickStack to external agents and AI tools.
We also [announced ClickStack Cloud](https://clickhouse.com/blog/clickstack-cloud-private-preview), now in private preview. ClickStack Cloud is a fully managed, serverless observability platform built on ClickHouse. Instead of managing collectors, ingestion infrastructure, scaling policies, storage, or schema tuning, teams simply send OpenTelemetry data to a managed endpoint and immediately begin exploring logs, metrics, and traces through the ClickStack UI. If you want ClickHouse-powered observability without operating the underlying infrastructure, ClickStack Cloud is designed to be the default path.
## New contributors
As always, thank you to our new contributors, and welcome to the community. Whether you're opening pull requests, filing issues, sharing ideas, or helping others, every contribution makes the project better for everyone.
[dewly-justice](https://github.com/dewly-justice), [doyong365](https://github.com/doyong365), [dvjn](https://github.com/dvjn), [jordan-simonovski](https://github.com/jordan-simonovski), [nohupped](https://github.com/nohupped)
## Dashboard actions
Tables are often the starting point of an investigation rather than the end of one. A chart showing error counts by service or latency by endpoint becomes much more useful when users can move directly from a row to the next step in their workflow. Until now, clicking a table row could only open a search using the row's group-by values as filters. While convenient, it was difficult to control where users landed or how context was carried across.
This month, we introduced configurable dashboard actions for table tiles. Row clicks can now open a search against a specific source with a custom `WHERE` clause, both of which support Handlebars templates that reference values from the selected row. Alternatively, rows can link directly to other dashboards, enabling drilldown workflows that move users from high-level operational views to more focused dashboards.
The feature is designed to behave like a normal web link. Rows with configured actions are rendered as `` elements, allowing users to preview destinations before clicking. After a short hover delay, ClickStack displays a tooltip describing the destination, such as opening a search or navigating to a dashboard. Standard browser behaviors also work as expected, including opening links in a new tab, copying link addresses, and previewing destinations in the browser status bar. Dashboard and source references are also remapped automatically during import and export, allowing linked dashboards to move cleanly between environments without manual updates.
## Custom color palettes for number visualizations
Number tiles are often the first thing users look at on a dashboard, but until now, they could only communicate a value and a label. Whether a number represented a healthy behavior, a warning sign, or an ongoing incident was left to the viewer to interpret.
Number tiles now support custom colors and threshold-based formatting. A tile can be assigned a fixed color or configured with ordered thresholds that automatically change its appearance as values cross defined boundaries. This makes it possible to highlight KPIs, error rates, latency targets, and other operational signals directly within the dashboard without requiring users to interpret the numbers themselves.
These settings are available across the UI, external dashboards API, and MCP dashboard tools, ensuring that programmatically generated dashboards can use the same visual cues as manually created ones. We also improved the rendering behavior of number tiles by automatically scaling text when space is limited, preventing values from overflowing smaller dashboard layouts.
## Experimental PromQL support
ClickHouse has always been a natural fit for logs, traces, and event-based metrics. Wide events compress extremely well in a column-oriented database, and ClickStack's metrics experience has historically been built around that model. At the same time, we recognize that some workloads come with their own established query languages and operational practices. For Prometheus users, that language is PromQL.
Over the last few months, we've been exploring what native PromQL support could look like inside ClickStack. Rather than forcing teams to move away from existing workflows, the goal is to allow Prometheus-style metrics to be queried using the language many engineers already know, while keeping those workflows alongside logs and traces in a single interface.
The implementation builds on the experimental ClickHouse TimeSeries Engine, which continues to [gain PromQL compatibility and functionality with each release](https://github.com/ClickHouse/ClickHouse/issues/57545#issuecomment-4246791048). When enabled, ClickStack provisions an `otel_metrics_ts` table, exposes Prometheus-compatible query endpoints, and adds a PromQL datasource and chart editor to the UI. Teams can either query metrics stored in ClickHouse through the TimeSeries Engine or configure an external Prometheus-compatible endpoint and proxy queries through ClickStack.
Teams running Prometheus-based metrics pipelines have always needed a separate system to query them, even when their logs and traces were already in ClickStack. We have started exploring what it would take to bring PromQL queries into the same interface, and this release ships the first iteration.
> This work is still highly experimental, and both the storage model and API surface are likely to evolve as the underlying TimeSeries Engine matures. Coverage continues to improve month by month, and we're actively evaluating how PromQL should fit into the broader ClickStack experience. If you're interested in testing the feature, collaborating on its direction, or have PromQL queries you'd like us to validate, we'd love to hear from you.
Event patterns are one of the fastest ways to make sense of large volumes of logs and traces. Rather than scrolling through thousands or millions of individual events, ClickStack automatically clusters similar messages together into a small number of representative groups.
This makes it easier to identify recurring errors, spot new issues, understand what drives volume spikes, and quickly summarize the types of events a service produces without writing regular expressions or maintaining parsing rules.
Until now, pattern analysis assumed that the text to be clustered came from a dedicated message or body column. If the value you wanted to analyze was stored elsewhere, such as a URL path, exception type, endpoint name, or custom attribute, there was no way to direct the pattern engine to use it.
You can now select any column as the source for pattern analysis directly from the event patterns view. This makes it possible to cluster and summarize a much wider range of observability data, opening up new workflows beyond traditional log message analysis.
## Browser RUM experience
Many ClickStack users are already instrumenting their frontends with the HyperDX Browser SDK or another OpenTelemetry browser agent, collecting session replays, page performance metrics, JavaScript errors, and user interaction data. Until now, those users were left to build their own dashboards from scratch. One of the most common requests has been for a browser observability experience similar to the [out-of-the-box Kubernetes dashboards](https://youtu.be/winI7256Ejk?si=EcD_2GkHZYlqGFc_) that ship with ClickStack.
This month, we're introducing a Browser RUM dashboard template in the dashboard gallery. The template provides a ready-made starting point for monitoring frontend applications, covering the areas most teams care about first: Core Web Vitals, page load performance, frontend errors, traffic breakdowns, and user experience metrics.
The dashboard includes p75 Core Web Vitals metrics such as LCP, INP, and CLS, page load percentiles, long task counts, JavaScript exceptions, API failures, and traffic segmented by dimensions such as URL, browser, country, and device size. Like other ClickStack templates, it includes dashboard-level filters and drilldown workflows, allowing teams to move from a high-level view of application health into specific pages, environments, versions, or user segments without modifying individual tiles.
The template requires a `rum.sessionId` resource attribute on incoming spans. Applications instrumented using the HyperDX Browser SDK emit this automatically.
## Fit Y-axis to data
By default, line charts anchor the Y-axis at zero. That's often the right choice when comparing absolute values, but it can make smaller changes difficult to see when a metric operates within a narrow range. Latency, CPU utilization, cache hit rates, and similar signals can end up looking almost flat despite meaningful variation over time.
A new **Fit Y-axis to Data** display option allows line charts to scale the Y-axis to the visible range of the data instead. This makes trends, regressions, and short-term fluctuations much easier to spot while leaving the underlying query and data unchanged.
## Improvements to the trace view
The trace view continues to be one of the most heavily used parts of ClickStack, and we've received a lot of feedback from users, particularly those migrating from Jaeger, where navigating and understanding traces is the core observability workflow. As a result, we've invested heavily in the trace experience over the last month, focusing on making investigations faster and reducing the amount of navigation required when working with larger traces.
The most visible change is a redesigned layout. Previously, selecting a span would display its details beneath the waterfall view, often pushing important information below the viewport. The trace view now uses a split-pane layout, with the waterfall on the left and span details appearing alongside it on the right. This makes it possible to inspect spans while keeping the full trace structure visible at the same time.
We've also added a full-screen mode for traces, giving users more room to explore complex traces with hundreds or thousands of spans. Span details now include dedicated Overview, Column Values, and Infrastructure tabs, with Kubernetes-specific infrastructure information automatically shown when available. Alongside the layout changes, we reworked timeline rendering to provide more consistent axis labels and spacing across different zoom levels, making trace navigation smoother regardless of trace size.
## Service map improvements
Since launching the service map last year, we've spent a lot of time gathering feedback from users and understanding how service maps fit into real investigation workflows. The challenge has never been drawing a graph, but making the graph useful enough to become the place engineers start when something goes wrong, while also ensuring it remains efficient to compute at ClickHouse scale.
This month, we significantly expanded the information available within the service map. In addition to request counts and error rates, services now display throughput and p50, p95, and p99 latency, making it much easier to identify overloaded or degraded services directly from the topology view.
We also introduced filtering and focusing controls to make the map more practical for larger environments. Users can filter services using a service selector or a Lucene/SQL filter expression, reducing noise and narrowing the graph to the systems they care about. A new Focus action allows any service to become the center of the map, showing only that service and its immediate dependencies. For organizations with dozens or hundreds of services, these additions make it much easier to move from a broad architectural view to a targeted investigation.
## MCP server tool investments
Since announcing the ClickStack MCP server at Open House, we've continued expanding both the breadth and depth of its observability capabilities. While generic SQL interfaces are powerful, we continue to see that AI agents perform significantly better when given access to higher-level observability primitives rather than being forced to reconstruct complex investigative workflows from raw queries. The ClickStack MCP server exposes those workflows directly, allowing agents to reason about traces, patterns, anomalies, dashboards, and operational artifacts using tools designed specifically for observability.
This month, we added new tools for trace analysis, event comparison, source discovery, and dashboard management. Agents can now retrieve trace waterfalls and breakdowns, compare event distributions across time windows, inspect source schemas and sample data, and create or modify dashboards directly through the MCP interface. We also introduced a denoise option for search workflows, helping agents focus on meaningful signals by suppressing repetitive high-frequency events that would otherwise dominate investigations.
We've also invested in an [internal evaluation framework](https://github.com/hyperdxio/hyperdx/pull/2414) that benchmarks investigation quality, tool usage, and response accuracy, helping us identify where agents struggle and measure improvements over time.
## Direct read optimization for accelerating attribute search
One of the bigger performance improvements we introduced in April was the direct-read optimization for OpenTelemetry attribute searches. OpenTelemetry stores much of its metadata inside flexible attribute maps, which are great for schema flexibility but traditionally more expensive to search efficiently. The optimization allows ClickStack to rewrite attribute filters into a form that aligns directly with ClickHouse text indexes, significantly reducing the amount of data that needs to be read during search operations. In our benchmarks, affected queries ran between 1.4x and 10x faster depending on the workload.
When we first introduced the feature, it was opt-in because it depended on companion materialized columns, which added storage overhead. Following improvements in ClickHouse that allow the same approach to work with ALIAS columns (no storage overhead), that limitation has now been removed, and the optimization is enabled by default for logs and traces sources. No configuration is required for most deployments. For a deeper explanation of how the optimization works, the benchmarks behind it, and the ClickHouse indexing techniques involved, see our [April update](/blog/whats-new-in-clickstack-april-2026#direct-read-optimization-for-opentelemetry-attribute-maps).
## Scoped dashboard filters
Dashboard-level filters are a powerful way to make dashboards interactive, but they were originally designed with single-source dashboards in mind. As more users began combining logs, traces, metrics, and custom sources into a single view, a common problem emerged: filters that made sense for one source were often meaningless for another. A filter on `SpanName`, for example, might be useful for a trace chart while causing a logs chart to return empty or unexpected results.
Dashboard filters can now be scoped to specific sources. When a filter has a source scope configured, only tiles using those sources will receive the filter. All other tiles ignore it entirely. This makes it much easier to build multi-source dashboards where each filter only affects the charts it was intended for, without introducing confusing interactions elsewhere. Existing dashboards continue to behave as before, with unscoped filters applying globally across all tiles.
## Little but useful things
### Sections for data sources
As ClickStack deployments grow, source lists can become difficult to navigate. Sources can now be assigned to sections, allowing related sources to be grouped together and searched more easily. Whether you're separating environments, teams, or applications, it's now much easier to find the source you're looking for.
### Editable filter pills
Search filters can now be edited directly in place. Instead of removing a filter and recreating it from scratch, clicking a filter pill opens an inline editor, making it much faster to refine searches during an investigation.
### Source field suggestions
Source configuration now suggests matching columns from the connected source when mapping fields such as timestamps, log levels, and message bodies, making new source setup quicker and reducing configuration errors.
### Filters included in dashboard exports
Dashboard exports now preserve active dashboard-level filters, ensuring exported dashboards capture the full state of the view rather than just the underlying tiles. Validation on import has also been added.
### Table of Contents for dashboards
Large dashboards now include a right-hand Table of Contents showing all sections, making navigation significantly easier. Sections can also be expanded or collapsed in bulk.
### Compatible filters preserved when switching sources
When changing a tile to use a different source, ClickStack now retains any filters that remain valid for the new source and only removes filters that reference unavailable columns.
## What's New in pg_clickhouse v0.3.2: Postgres 19, TLS, Regex, and Memory
Last week we shipped the latest version of [pg_clickhouse](https://clickhouse.com/docs/cloud/managed-postgres/extensions/pg_clickhouse), the interface for querying [ClickHouse](https://clickhouse.com/clickhouse) from [Postgres](https://www.postgresql.org/). As a minor update, [v0.3.2](https://github.com/ClickHouse/pg_clickhouse/releases/tag/v0.3.2) requires no reload, restart, or `ALTER EXTENSION UPDATE`, and we've upgraded all of the ClickHouse Cloud instances. Your next connection to the database will load the latest and greatest.
Despite the minor version increment, this release significantly improves pg_clickhouse in four key areas: Postgres 19, TLS connections, regular expression pushdown, and memory consumption.
## Postgres 19
The topline change? Support for [PostgreSQL 19 Beta1](https://www.postgresql.org/about/news/postgresql-19-beta-1-released-3313/). The new Postgres version required relatively minor revisions to the pg_clickhouse source code to take advantage of tuple and array optimizations, remove old typedefs, add new headers, and some test outputs. And with that, we'll be ready for the final Postgres release this fall and ship day one on Manged Postgres for ClickHouse.
## TLS connections
[pg_clickhouse](https://clickhouse.com/docs/cloud/managed-postgres/extensions/pg_clickhouse) has supported TLS connections since its first release, but [v0.3.2](https://github.com/ClickHouse/pg_clickhouse/releases/tag/v0.3.2) introduces a couple of new `CREATE SERVER` options:
- `secure` specifies the security requirement for a connection: `on` (force TLS), `off` (force plaintext), or `auto` (cloud-host/port heuristic, the default). Thanks to [Andrey Borodin](https://x4mmm.medium.com) for the inspiring [pull request](https://github.com/ClickHouse/pg_clickhouse/pull/227).
- `min_tls_version` specifies a minimum TLS protocol version: `TLSv1`, `TLSv1.1`, `TLSv1.2`, or `TLSv1.3`. It defaults to the TLS library's own minimum.
## Regular expressions
More in-depth exploration of the differing behaviors of regular expression flags revealed errors in our pushdown logic, now repaired. The [Postgres flags](https://www.postgresql.org/docs/current/functions-matching.html#POSIX-EMBEDDED-OPTIONS-TABLE) now push down to ClickHouse as follows:
| Flag | As | Notes |
| ---- | ----- | -------------------------------------------------------------- |
| `i` | `i` | case-insensitive matching |
| `m` | `m-s` | `^` and `$` match begin/end line in addition to begin/end text |
| `n` | `m-s` | Postgres alias for `m` |
| `p` | `-s` | don't let `.` and `[^x]` match `\n` |
| `s` | `s` | let `.` and `[^x]` match `\n` |
| `t` | | tight syntax, ignored |
| `w` | `m` | inverse partial newline-sensitive matching |
The [documentation](https://pgxn.org/dist/pg_clickhouse/doc/pg_clickhouse.html#Regular.Expressions) also notes the variation in the behaviors of `m` and `p`, in which Postgres prevents negated character classes (`[^xyz]`) from matching a newline, while the ClickHouse equivalents do not. Be sure to carefully test regular expressions that use character classes.
## Memory consumption
A couple of customer queries revealed some memory consumption issues.
One was triggered by using unbuffered queries with the HTTP driver. Such a configuration has not been recommended or the default since v0.1.10, so should be quite rare.
The other issue arose when a foreign scan repeatedly re-scanned, as in a nested-loop join with a parameterized inner foreign scan --- a fairly typical plan. Be sure to upgrade if you notice memory ballooning while querying a foreign table.
## And more
Other changes worth mentioning:
- Added the `compression` option to `CREATE SERVER` to enable ClickHouse native protocol compression for query results and `INSERT` data
- Added mapping to push down `regexp_match()` when its regex argument contains no capturing groups
- Fixed a bug where `ANY()` with an empty array (`WHERE x = ANY('{}')`) produced an error in Clickhouse prior to version 25
Download from the usual locations:
- [PGXN](https://pgxn.org/dist/pg_clickhouse/0.3.2/)
- [GitHub](https://github.com/ClickHouse/pg_clickhouse/releases/tag/v0.3.2)
- [Docker](https://github.com/ClickHouse/pg_clickhouse/pkgs/container/pg_clickhouse)
### Try Postgres managed by ClickHouse
ClickHouse + Postgres has become the unified data stack for applications that scale. With Managed Postgres now available in ClickHouse Cloud, this stack is a day-1 decision.
[Get access](https://clickhouse.com/cloud/postgres?loc=blog-cta-994-try-postgres-managed-by-clickhouse-get-access&utm_blogctaid=994)
## What's Changed
* chore(deps): bump neon-init to 0.19.0 by @andrelandgraf in https://github.com/neondatabase/neonctl/pull/583
* chore: release neonctl@v2.27.0 by @neonctl-semantic-release[bot] in https://github.com/neondatabase/neonctl/pull/584
**Full Changelog**: https://github.com/neondatabase/neonctl/compare/v2.26.8...v2.27.0
## Release v26.3.14.49-lts
## What's Changed
* refactor(bootstrap): consume shared core from neon-init/bootstrap by @andrelandgraf in https://github.com/neondatabase/neonctl/pull/577
* chore: release neonctl@v2.26.8 by @neonctl-semantic-release[bot] in https://github.com/neondatabase/neonctl/pull/582
**Full Changelog**: https://github.com/neondatabase/neonctl/compare/v2.26.7...v2.26.8
## Release v26.5.3.52-stable
## What's Changed
* fix(env): drop removed NEON_STORAGE_FORCE_PATH_STYLE from owned env keys by @andrelandgraf in https://github.com/neondatabase/neonctl/pull/575
* fix(env): drop removed NEON_STORAGE_REGION from owned env keys by @andrelandgraf in https://github.com/neondatabase/neonctl/pull/576
* chore(deps): bump @neondatabase/env to latest by @guillaumervls in https://github.com/neondatabase/neonctl/pull/579
* chore: release neonctl@v2.26.7 by @neonctl-semantic-release[bot] in https://github.com/neondatabase/neonctl/pull/581
**Full Changelog**: https://github.com/neondatabase/neonctl/compare/v2.26.6...v2.26.7
## 9.21.0
This is a minor release adding new features and bug fixes. There are no breaking changes; upgrading from 9.20.x is a drop-in replacement.
## π Highlights
### Zero-copy `GetToBuffer` / `SetFromBuffer`
Two new `StringCmdable` methods let callers read and write Redis string values directly into and from pre-allocated byte buffers, eliminating the per-call payload allocation that `Get`/`Set` incur:
```go
GetToBuffer(ctx, key, buf) *ZeroCopyStringCmd // reads into buf; ZeroCopyStringCmd { Val() int; Bytes() []byte; Result() (int, error) }
SetFromBuffer(ctx, key, buf) *StatusCmd
```
`GetToBuffer` decodes the bulk reply straight into the caller-owned `buf` (no intermediate allocation); a buffer that is too small returns an error after draining the payload, so the connection stays aligned for the next reply. `SetFromBuffer` is provided for API symmetry β it dispatches to the same `[]byte` writer path as `Set(ctx, key, buf, 0)` and produces byte-identical output on the wire. Available on `*Client`, `*ClusterClient`, `*Ring`, `*Conn` and `Pipeliner`.
([#3834](https://github.com/redis/go-redis/pull/3834)) by [@ndyakov](https://github.com/ndyakov)
### Explicit `LIMIT 0` for stream trimming
Redis treats `XTRIM`/`XADD` approximate-trim (`~`) `LIMIT 0` as "disable the trimming effort cap entirely", which differs from omitting `LIMIT` (the implicit `100 * stream-node-max-entries` default). The command builders previously only emitted `LIMIT` when `limit > 0`, so callers could never send an explicit `LIMIT 0`. Following the `KeepTTL = -1` precedent, the new `XTrimLimitDisabled = -1` sentinel now emits an explicit `LIMIT 0`; `limit == 0` keeps the historical no-`LIMIT` behavior, so existing callers produce byte-identical commands.
([#3848](https://github.com/redis/go-redis/pull/3848)) by [@TheRealMal](https://github.com/TheRealMal)
## β¨ New Features
- **Zero-copy buffer string commands**: new `GetToBuffer` / `SetFromBuffer` on `StringCmdable` and the `ZeroCopyStringCmd` result type, reading/writing string values into caller-owned buffers without per-call payload allocation ([#3834](https://github.com/redis/go-redis/pull/3834)) by [@ndyakov](https://github.com/ndyakov)
- **`XTrimLimitDisabled` sentinel**: `XTRIM`/`XADD` approximate trimming can now send an explicit `LIMIT 0` to disable the trim effort cap, via the new `XTrimLimitDisabled = -1` sentinel ([#3848](https://github.com/redis/go-redis/pull/3848)) by [@TheRealMal](https://github.com/TheRealMal)
- **PubSub health-check timeouts**: `channel.initHealthCheck` now bounds the `Ping` it issues with a fresh per-check timeout context (the exported `pingTimeout` / `reconnectTimeout`) instead of `context.TODO()`, so a stuck health-check Ping can no longer block indefinitely ([#3819](https://github.com/redis/go-redis/pull/3819)) by [@abdellani](https://github.com/abdellani)
- **Skip redundant `UNWATCH` in `Tx.Close`**: a transaction now tracks whether a `WATCH` is still active (`watchArmed`) and only issues `UNWATCH` on `Close` when it is, removing an extra round trip on the common `WATCH`/.../`EXEC` and no-key `Watch` paths while never returning a connection to the pool with an active watch ([#3854](https://github.com/redis/go-redis/pull/3854)) by [@fcostaoliveira](https://github.com/fcostaoliveira)
## π Bug Fixes
- **`maintnotifications` `ModeAuto` fail-open**: `ModeAuto` now stays fail-open when the server does not support maintenance notifications β connections are retired and tracking is guarded during downgrade so the client keeps working instead of erroring ([#3853](https://github.com/redis/go-redis/pull/3853)) by [@terrorobe](https://github.com/terrorobe)
## π₯ Contributors
We'd like to thank all the contributors who worked on this release!
[@abdellani](https://github.com/abdellani), [@fcostaoliveira](https://github.com/fcostaoliveira), [@ndyakov](https://github.com/ndyakov), [@terrorobe](https://github.com/terrorobe), [@TheRealMal](https://github.com/TheRealMal)
## How Alien uses Turso to make BYOC deployments reliable
Case study: How Alien leverages Turso for reliable bring-your-own-cloud deployments.
## Web console now supports Postgres
We've released several improvements to the PlanetScale web console:
- Now available for Postgres databases (beta, please send us your feedback)
- Table and column name autocomplete
- Ability to export query results to csv, json or sql format
- You can now choose whether to run queries against your primary or a replica
**Read more:** [Postgres web console](https://planetscale.com/docs/postgres/web-console) or [Vitess web console](https://planetscale.com/docs/vitess/web-console)