{"collection":{"slug":"data-engineering","name":"Data & Analytics Engineering"},"digest":{"id":"cwd_Qt8yiVC1WJstZ2VYGeWE3","weekStart":"2026-09-28","title":"ClickHouse's busy week: cloud identity, OneLake, and Postgres memory safety","intro":"ClickHouse shipped a torrent of stable and LTS patches while rolling out SCIM provisioning for Cloud, deeper Microsoft Fabric and OneLake integration, and a hardened memory-safety story for its Postgres extensions. Dagster gave partitioned automation a performance boost and introduced a mobile UI, and pg_clickhouse and chdb tightened type and encoding handling.","body":"### ClickHouse bets big on Microsoft\n\nThe week's headline is ClickHouse's push into Microsoft's data ecosystem. A [native Fabric workload](/release/rel_mine5Arp8Tx4H2zJqcP0y-clickhouse-workload-for-microsoft-fabric-sub-second-analytics-on-onelake-now-in) brings sub-second analytics to OneLake data in public preview, backed by dedicated ClickHouse Cloud compute for agents, dashboards, and applications. Alongside it, ClickHouse can now [write Apache Iceberg tables directly to OneLake](/release/rel_yTgFKdOy9eTUJwLhy3iJt-clickhouse-now-writes-apache-iceberg-tables-to-microsoft-onelake), making the lake a first-class landing zone for analytical workloads. Both arrive under a broader [expanded Microsoft collaboration](/release/rel_B44DgoKdeHP7YMKEXTWIM-clickhouse-expands-collaboration-with-microsoft-bringing-fabric-integration) that adds OneLake read and write capabilities and makes ClickHouse BYOC available in the Microsoft Marketplace.\n\n### Cloud identity and the operational layers\n\nOn the managed side, ClickHouse Cloud picked up [SCIM 2.0 provisioning](/release/rel_PPH6QCzmiAOSDTeRK_leN-introducing-scim-provisioning-in-clickhouse-cloud), so an identity provider can create, update, and deprovision organization members and sync group membership to ClickHouse roles automatically — a meaningful change for teams wiring enterprise SSO into their analytics stack. The blog also dug into the internals of its managed Postgres: a piece on [why direct I/O backs its backups](/release/rel_1GCsLCudCEl9ZCsB_QpL1-what-is-direct-i-o-and-why-does-clickhouse-managed-postgres-use-it-for-backups) explains how stripe-sized reads keep backups fast without polluting the page cache and hurting query latency, and a [recursive query benchmark](/release/rel_BKAfWvyet45-jnEAHedwt-can-your-postgres-survive-a-bad-query) compares whether Postgres clusters survive a memory-exhausting query against Cloud SQL, PlanetScale, and Amazon RDS.\n\n### Extensions, encoding, and agent memory\n\nThe Postgres bridge saw real polish: the [latest pg_clickhouse and chdb releases](/release/rel_ySJCyIGwVKAZ1GOkGTyx_-pg-clickhouse-chdb-updates-encoding-nesting-and-types) improve character encoding, interval handling, nested data, JSON, and type mappings between the two engines. ClickHouse also published a [memory-safety walkthrough for its Postgres extensions](/release/rel_aGao1Xzu-DFjowz6YBWAp-memory-safety-for-postgres-extensions-in-c-c), covering the challenges of mixing C and C++ — clean language boundaries and isolated helper processes — which pairs naturally with the extension work above. For the agent crowd, the [chDB Durable Layer](/release/rel_BlsUThIaKJ7WNoNWKzsdw-chdb-durable-layer-for-agent-memory) keeps agent memory fast to query locally while making its analytical state recoverable across laptops, CI jobs, and short-lived sandboxes.\n\nAnd behind all of this, ClickHouse kept its release trains moving: a [steady stream of stable and LTS updates](/release/rel_tTfB7kZyhqM_Gm_ztj53E-release-v26-9-10-4-stable) across the 26.3, 26.7, 26.8, and 26.9 lines, spanning [dozens of patches](/release/rel_UmSL-wPsMAVXvmjnNe2s0-release-v26-7-21-2-stable) shipped throughout the week.\n\n### Dagster goes mobile, Tinybird sets a kernel floor\n\nDagster's week centered on scale and access: its [partitioned declarative automation got faster](/release/rel_0D_JxX9UKF9HQO1AULo-h-1-13-25-core-0-29-25-libraries) for assets with long materialization histories, and the UI now has [a proper mobile layout](/release/rel_0D_JxX9UKF9HQO1AULo-h-1-13-25-core-0-29-25-libraries) with navigation and mobile versions of Home, Jobs, Automation, and the Asset Catalog. Tinybird, meanwhile, flagged a [4KiB page-size requirement for arm64](/release/rel_nFcQTJ8qVLxuc_ANUJ2Fd-tinybird-local-on-arm64-requires-a-4kib-page-size-kernel) Tinybird Local images — most distributions are fine, but kernels with larger pages won't start ClickHouse, so arm64 users may need to switch flavors or fall back to amd64.","releaseIds":["rel_mine5Arp8Tx4H2zJqcP0y","rel_yTgFKdOy9eTUJwLhy3iJt","rel_B44DgoKdeHP7YMKEXTWIM","rel_PPH6QCzmiAOSDTeRK_leN","rel_1GCsLCudCEl9ZCsB_QpL1","rel_BKAfWvyet45-jnEAHedwt","rel_ySJCyIGwVKAZ1GOkGTyx_","rel_aGao1Xzu-DFjowz6YBWAp","rel_BlsUThIaKJ7WNoNWKzsdw","rel_tTfB7kZyhqM_Gm_ztj53E","rel_UmSL-wPsMAVXvmjnNe2s0","rel_0D_JxX9UKF9HQO1AULo-h","rel_nFcQTJ8qVLxuc_ANUJ2Fd"],"releaseCount":34,"generatedAt":"2026-10-05T06:20:30.820Z","releases":[{"id":"rel_mine5Arp8Tx4H2zJqcP0y","title":"ClickHouse Workload for Microsoft Fabric: sub-second analytics on OneLake, now in public preview","path":"/release/rel_mine5Arp8Tx4H2zJqcP0y-clickhouse-workload-for-microsoft-fabric-sub-second-analytics-on-onelake-now-in","url":"https://clickhouse.com/blog/clickhouse-for-microsoft-fabric","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_yTgFKdOy9eTUJwLhy3iJt","title":"ClickHouse now writes Apache Iceberg tables to Microsoft OneLake","path":"/release/rel_yTgFKdOy9eTUJwLhy3iJt-clickhouse-now-writes-apache-iceberg-tables-to-microsoft-onelake","url":"https://clickhouse.com/blog/clickhouse-now-writes-to-microsoft-onelake","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_B44DgoKdeHP7YMKEXTWIM","title":"ClickHouse expands collaboration with Microsoft, bringing Fabric integration, deeper OneLake interoperability, and enterprise deployment flexibility","path":"/release/rel_B44DgoKdeHP7YMKEXTWIM-clickhouse-expands-collaboration-with-microsoft-bringing-fabric-integration","url":"https://clickhouse.com/blog/clickhouse-expands-collaboration-with-microsoft","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_PPH6QCzmiAOSDTeRK_leN","title":"Introducing SCIM Provisioning in ClickHouse Cloud","path":"/release/rel_PPH6QCzmiAOSDTeRK_leN-introducing-scim-provisioning-in-clickhouse-cloud","url":"https://clickhouse.com/blog/introducing-scim-provisioning-in-clickhouse-cloud","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_1GCsLCudCEl9ZCsB_QpL1","title":"What is direct I/O, and why does ClickHouse Managed Postgres use it for backups?","path":"/release/rel_1GCsLCudCEl9ZCsB_QpL1-what-is-direct-i-o-and-why-does-clickhouse-managed-postgres-use-it-for-backups","url":"https://clickhouse.com/blog/direct-io-managed-postgres-backups","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_BKAfWvyet45-jnEAHedwt","title":"Can your Postgres survive a bad query?","path":"/release/rel_BKAfWvyet45-jnEAHedwt-can-your-postgres-survive-a-bad-query","url":"https://clickhouse.com/blog/can-your-postgres-survive-a-bad-query","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_ySJCyIGwVKAZ1GOkGTyx_","title":"pg_clickhouse & chdb updates: Encoding, nesting, and types","path":"/release/rel_ySJCyIGwVKAZ1GOkGTyx_-pg-clickhouse-chdb-updates-encoding-nesting-and-types","url":"https://clickhouse.com/blog/pg_clickhouse-chdb","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_aGao1Xzu-DFjowz6YBWAp","title":"Memory safety for Postgres extensions in C/C++","path":"/release/rel_aGao1Xzu-DFjowz6YBWAp-memory-safety-for-postgres-extensions-in-c-c","url":"https://clickhouse.com/blog/memory-safety-postgres-extensions-c-cpp","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_BlsUThIaKJ7WNoNWKzsdw","title":"chDB Durable Layer for agent memory","path":"/release/rel_BlsUThIaKJ7WNoNWKzsdw-chdb-durable-layer-for-agent-memory","url":"https://clickhouse.com/blog/chdb-durable-layer-for-agent-memory","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_tTfB7kZyhqM_Gm_ztj53E","title":"Release v26.9.10.4-stable","path":"/release/rel_tTfB7kZyhqM_Gm_ztj53E-release-v26-9-10-4-stable","url":"https://github.com/ClickHouse/ClickHouse/releases/tag/v26.9.10.4-stable","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_UmSL-wPsMAVXvmjnNe2s0","title":"Release v26.7.21.2-stable","path":"/release/rel_UmSL-wPsMAVXvmjnNe2s0-release-v26-7-21-2-stable","url":"https://github.com/ClickHouse/ClickHouse/releases/tag/v26.7.21.2-stable","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_0D_JxX9UKF9HQO1AULo-h","title":"1.13.25 (core) / 0.29.25 (libraries)","path":"/release/rel_0D_JxX9UKF9HQO1AULo-h-1-13-25-core-0-29-25-libraries","url":"https://github.com/dagster-io/dagster/releases/tag/1.13.25","org":{"slug":"dagster","name":"Dagster","avatarUrl":"https://media.releases.sh/orgs/dagster.jpg","githubHandle":"dagster-io"},"product":null,"importance":null},{"id":"rel_nFcQTJ8qVLxuc_ANUJ2Fd","title":"Tinybird Local on arm64 requires a 4KiB page size kernel","path":"/release/rel_nFcQTJ8qVLxuc_ANUJ2Fd-tinybird-local-on-arm64-requires-a-4kib-page-size-kernel","url":"https://www.tinybird.co/docs/changelog/2026-10-02-october-week-one","org":{"slug":"tinybird","name":"Tinybird","avatarUrl":"https://media.releases.sh/orgs/tinybird.png","githubHandle":"tinybirdco"},"product":null,"importance":null}],"sections":[{"heading":"ClickHouse bets big on Microsoft","anchor":"clickhouse-bets-big-on-microsoft","lede":"The week's headline is ClickHouse's push into Microsoft's data ecosystem.","releaseIds":["rel_mine5Arp8Tx4H2zJqcP0y","rel_yTgFKdOy9eTUJwLhy3iJt","rel_B44DgoKdeHP7YMKEXTWIM"],"releases":[{"id":"rel_mine5Arp8Tx4H2zJqcP0y","title":"ClickHouse Workload for Microsoft Fabric: sub-second analytics on OneLake, now in public preview","path":"/release/rel_mine5Arp8Tx4H2zJqcP0y-clickhouse-workload-for-microsoft-fabric-sub-second-analytics-on-onelake-now-in","url":"https://clickhouse.com/blog/clickhouse-for-microsoft-fabric","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_yTgFKdOy9eTUJwLhy3iJt","title":"ClickHouse now writes Apache Iceberg tables to Microsoft OneLake","path":"/release/rel_yTgFKdOy9eTUJwLhy3iJt-clickhouse-now-writes-apache-iceberg-tables-to-microsoft-onelake","url":"https://clickhouse.com/blog/clickhouse-now-writes-to-microsoft-onelake","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_B44DgoKdeHP7YMKEXTWIM","title":"ClickHouse expands collaboration with Microsoft, bringing Fabric integration, deeper OneLake interoperability, and enterprise deployment flexibility","path":"/release/rel_B44DgoKdeHP7YMKEXTWIM-clickhouse-expands-collaboration-with-microsoft-bringing-fabric-integration","url":"https://clickhouse.com/blog/clickhouse-expands-collaboration-with-microsoft","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null}]},{"heading":"Cloud identity and the operational layers","anchor":"cloud-identity-and-the-operational-layers","lede":"On the managed side, ClickHouse Cloud picked up SCIM 2.0 provisioning, so an identity provider can create, update, and deprovision organization members and sync group membership to ClickHouse roles automatically — a meaningful change for teams wiring enterprise SSO into their analytics stack.","releaseIds":["rel_PPH6QCzmiAOSDTeRK_leN","rel_1GCsLCudCEl9ZCsB_QpL1","rel_BKAfWvyet45-jnEAHedwt"],"releases":[{"id":"rel_PPH6QCzmiAOSDTeRK_leN","title":"Introducing SCIM Provisioning in ClickHouse Cloud","path":"/release/rel_PPH6QCzmiAOSDTeRK_leN-introducing-scim-provisioning-in-clickhouse-cloud","url":"https://clickhouse.com/blog/introducing-scim-provisioning-in-clickhouse-cloud","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_1GCsLCudCEl9ZCsB_QpL1","title":"What is direct I/O, and why does ClickHouse Managed Postgres use it for backups?","path":"/release/rel_1GCsLCudCEl9ZCsB_QpL1-what-is-direct-i-o-and-why-does-clickhouse-managed-postgres-use-it-for-backups","url":"https://clickhouse.com/blog/direct-io-managed-postgres-backups","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_BKAfWvyet45-jnEAHedwt","title":"Can your Postgres survive a bad query?","path":"/release/rel_BKAfWvyet45-jnEAHedwt-can-your-postgres-survive-a-bad-query","url":"https://clickhouse.com/blog/can-your-postgres-survive-a-bad-query","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null}]},{"heading":"Extensions, encoding, and agent memory","anchor":"extensions-encoding-and-agent-memory","lede":"The Postgres bridge saw real polish: the latest pg_clickhouse and chdb releases improve character encoding, interval handling, nested data, JSON, and type mappings between the two engines.","releaseIds":["rel_ySJCyIGwVKAZ1GOkGTyx_","rel_aGao1Xzu-DFjowz6YBWAp","rel_BlsUThIaKJ7WNoNWKzsdw","rel_tTfB7kZyhqM_Gm_ztj53E","rel_UmSL-wPsMAVXvmjnNe2s0"],"releases":[{"id":"rel_ySJCyIGwVKAZ1GOkGTyx_","title":"pg_clickhouse & chdb updates: Encoding, nesting, and types","path":"/release/rel_ySJCyIGwVKAZ1GOkGTyx_-pg-clickhouse-chdb-updates-encoding-nesting-and-types","url":"https://clickhouse.com/blog/pg_clickhouse-chdb","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_aGao1Xzu-DFjowz6YBWAp","title":"Memory safety for Postgres extensions in C/C++","path":"/release/rel_aGao1Xzu-DFjowz6YBWAp-memory-safety-for-postgres-extensions-in-c-c","url":"https://clickhouse.com/blog/memory-safety-postgres-extensions-c-cpp","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_BlsUThIaKJ7WNoNWKzsdw","title":"chDB Durable Layer for agent memory","path":"/release/rel_BlsUThIaKJ7WNoNWKzsdw-chdb-durable-layer-for-agent-memory","url":"https://clickhouse.com/blog/chdb-durable-layer-for-agent-memory","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_tTfB7kZyhqM_Gm_ztj53E","title":"Release v26.9.10.4-stable","path":"/release/rel_tTfB7kZyhqM_Gm_ztj53E-release-v26-9-10-4-stable","url":"https://github.com/ClickHouse/ClickHouse/releases/tag/v26.9.10.4-stable","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null},{"id":"rel_UmSL-wPsMAVXvmjnNe2s0","title":"Release v26.7.21.2-stable","path":"/release/rel_UmSL-wPsMAVXvmjnNe2s0-release-v26-7-21-2-stable","url":"https://github.com/ClickHouse/ClickHouse/releases/tag/v26.7.21.2-stable","org":{"slug":"clickhouse","name":"ClickHouse","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse"},"product":null,"importance":null}]},{"heading":"Dagster goes mobile, Tinybird sets a kernel floor","anchor":"dagster-goes-mobile-tinybird-sets-a-kernel-floor","lede":"Dagster's week centered on scale and access: its partitioned declarative automation got faster for assets with long materialization histories, and the UI now has a proper mobile layout with navigation and mobile versions of Home, Jobs, Automation, and the Asset Catalog.","releaseIds":["rel_0D_JxX9UKF9HQO1AULo-h","rel_nFcQTJ8qVLxuc_ANUJ2Fd"],"releases":[{"id":"rel_0D_JxX9UKF9HQO1AULo-h","title":"1.13.25 (core) / 0.29.25 (libraries)","path":"/release/rel_0D_JxX9UKF9HQO1AULo-h-1-13-25-core-0-29-25-libraries","url":"https://github.com/dagster-io/dagster/releases/tag/1.13.25","org":{"slug":"dagster","name":"Dagster","avatarUrl":"https://media.releases.sh/orgs/dagster.jpg","githubHandle":"dagster-io"},"product":null,"importance":null},{"id":"rel_nFcQTJ8qVLxuc_ANUJ2Fd","title":"Tinybird Local on arm64 requires a 4KiB page size kernel","path":"/release/rel_nFcQTJ8qVLxuc_ANUJ2Fd-tinybird-local-on-arm64-requires-a-4kib-page-size-kernel","url":"https://www.tinybird.co/docs/changelog/2026-10-02-october-week-one","org":{"slug":"tinybird","name":"Tinybird","avatarUrl":"https://media.releases.sh/orgs/tinybird.png","githubHandle":"tinybirdco"},"product":null,"importance":null}]}]}}