{"collection":{"slug":"data-engineering","name":"Data & Analytics Engineering"},"digest":{"id":"cwd_c_U9teYi6bB6AGF0LkIce","weekStart":"2026-05-25","title":"ClickHouse’s conference week reshapes query execution and managed services","intro":"ClickHouse Open House 2026 delivered multi-stage distributed query execution, a managed Postgres beta, and serverless observability through ClickStack Cloud, while Dagster shipped its 1.13.7 release with improved schedule ownership and deeper Snowflake integration.","body":"### Multi-stage queries reshape large-scale analytics\n\nThe biggest engineering story this week is ClickHouse’s multi-stage distributed query execution, unveiled at the company’s Open House conference. [Multi-stage distributed execution](/release/rel_FSnjj2919tLx_frOKfHW5-introducing-multi-stage-distributed-query-execution-in-clickhouse-cloud) repartitions intermediate data between query stages, removing the bottlenecks that plague large joins and high-cardinality aggregations in distributed settings. Early TPC-H results show join-heavy queries running up to 3.4× faster, while aggregation scaling remains near-linear at 7.4× faster on eight nodes than one. This is a fundamental architectural change for ClickHouse Cloud, not a tuning knob — teams running complex analytical queries across many nodes will see the most impact.\n\n### Managed Postgres and serverless observability enter beta\n\nClickHouse’s conference also brought two significant managed service announcements. [Postgres managed by ClickHouse](/release/rel_iQMYiNh1DuC9YlfU87pqf-postgres-managed-by-clickhouse-is-now-in-beta) entered public beta, offering a fully managed, NVMe-backed Postgres service with native CDC into ClickHouse and a unified query layer via the pg_clickhouse extension. On the observability front, [ClickStack Cloud](/release/rel_jL4qoKF-moV0rhFAkhLP3-introducing-clickstack-cloud-serverless-observability-powered-by-clickhouse) launched as a serverless platform where teams send OpenTelemetry data to a managed endpoint and immediately explore logs, metrics, and traces without operating any infrastructure. The observability story deepened with [AI Notebooks and an MCP server](/release/rel_A8-dx3z5DbD1bI5USj_Yl-open-house-observability-announcements-mcp-server-ai-notebooks-and-clickstack), both now available in beta, alongside [CostBench](/release/rel_xjQq5-bGX-kDwgJEbKjoH-introducing-costbench-an-open-benchmark-for-data-warehouse-cost-performance), an open benchmark for comparing cloud data warehouse cost-performance across vendors.\n\n### Dagster tightens the governance and Snowflake loop\n\nWhile ClickHouse dominated with conference news, Dagster shipped [version 1.13.7](/release/rel_psdlA_9MgbuZeUI0M2eLM-1-13-7-core-0-29-7-libraries) with several practical improvements. The `build_schedule_from_partitioned_job` function now accepts an `owners` parameter, making schedule ownership explicit at definition time. The Fivetran component gained optional column-level metadata fetching for synced tables — a small but meaningful step for lineage tracking. Two accompanying blog posts framed Dagster’s positioning: one explained how [Dagster Compass powers self-service analytics at Brooklyn Data](/release/rel_7ILG_EPnlMdSkDWSrkuVI-how-dagster-compass-powers-brooklyn-data-s-self-service-analytics) by layering governance and business context on top of Snowflake, while another made the case that [Snowflake runs data, Dagster runs everything else](/release/rel_rPVb5P3sxInnK6NZNPDvM-snowflake-runs-your-data-dagster-runs-everything-else), covering how the orchestrator handles transformation, lineage, automation, and cost visibility across the broader platform.\n\n### dbt on agent infrastructure\n\nA single post from dbt Labs this week generated conversation: [what data infrastructure agents actually need](/release/rel_YrP17TS6Y87hhYJVo1MST-what-data-infrastructure-do-agents-need). The argument is that AI agents fail not because of weak models but because they run on infrastructure designed for batch training and human analytics — a claim that resonated with teams already experimenting with agentic workflows. It’s a short read that frames a growing tension between OLAP-oriented warehouses and the real-time, decision-making demands of autonomous systems.","releaseIds":["rel_FSnjj2919tLx_frOKfHW5","rel_iQMYiNh1DuC9YlfU87pqf","rel_jL4qoKF-moV0rhFAkhLP3","rel_A8-dx3z5DbD1bI5USj_Yl","rel_xjQq5-bGX-kDwgJEbKjoH","rel_psdlA_9MgbuZeUI0M2eLM","rel_7ILG_EPnlMdSkDWSrkuVI","rel_rPVb5P3sxInnK6NZNPDvM","rel_YrP17TS6Y87hhYJVo1MST"],"releaseCount":13,"generatedAt":"2026-07-11T16:38:58.577Z","releases":[{"id":"rel_FSnjj2919tLx_frOKfHW5","title":"Introducing multi-stage distributed query execution in ClickHouse Cloud","path":"/release/rel_FSnjj2919tLx_frOKfHW5-introducing-multi-stage-distributed-query-execution-in-clickhouse-cloud","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_iQMYiNh1DuC9YlfU87pqf","title":"Postgres managed by ClickHouse is now in beta","path":"/release/rel_iQMYiNh1DuC9YlfU87pqf-postgres-managed-by-clickhouse-is-now-in-beta","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_jL4qoKF-moV0rhFAkhLP3","title":"Introducing ClickStack Cloud: Serverless observability powered by ClickHouse","path":"/release/rel_jL4qoKF-moV0rhFAkhLP3-introducing-clickstack-cloud-serverless-observability-powered-by-clickhouse","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_A8-dx3z5DbD1bI5USj_Yl","title":"Open House observability announcements: MCP server, AI Notebooks, and ClickStack Cloud","path":"/release/rel_A8-dx3z5DbD1bI5USj_Yl-open-house-observability-announcements-mcp-server-ai-notebooks-and-clickstack","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_xjQq5-bGX-kDwgJEbKjoH","title":"Introducing CostBench: an open benchmark for data warehouse cost-performance","path":"/release/rel_xjQq5-bGX-kDwgJEbKjoH-introducing-costbench-an-open-benchmark-for-data-warehouse-cost-performance","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_psdlA_9MgbuZeUI0M2eLM","title":"1.13.7 (core) / 0.29.7 (libraries)","path":"/release/rel_psdlA_9MgbuZeUI0M2eLM-1-13-7-core-0-29-7-libraries","org":{"slug":"dagster","name":"Dagster"},"importance":null},{"id":"rel_7ILG_EPnlMdSkDWSrkuVI","title":"How Dagster Compass Powers Brooklyn Data's Self-Service Analytics","path":"/release/rel_7ILG_EPnlMdSkDWSrkuVI-how-dagster-compass-powers-brooklyn-data-s-self-service-analytics","org":{"slug":"dagster","name":"Dagster"},"importance":null},{"id":"rel_rPVb5P3sxInnK6NZNPDvM","title":"Snowflake Runs Your Data: Dagster Runs Everything Else","path":"/release/rel_rPVb5P3sxInnK6NZNPDvM-snowflake-runs-your-data-dagster-runs-everything-else","org":{"slug":"dagster","name":"Dagster"},"importance":null},{"id":"rel_YrP17TS6Y87hhYJVo1MST","title":"What data infrastructure do agents need?","path":"/release/rel_YrP17TS6Y87hhYJVo1MST-what-data-infrastructure-do-agents-need","org":{"slug":"dbt-labs","name":"dbt Labs"},"importance":null}]}}