{"slug":"data-engineering","name":"Data & Analytics Engineering","description":"Transform, orchestrate, and query data at scale — the modern data stack.","isFeatured":false,"dailySummaryEnabled":true,"weeklyDigestEnabled":true,"members":[{"kind":"org","slug":"dbt-labs","name":"dbt Labs","domain":"getdbt.com","avatarUrl":"https://media.releases.sh/orgs/dbt-labs.png","githubHandle":"dbt-labs","description":null},{"kind":"org","slug":"dagster","name":"Dagster","domain":"dagster.io","avatarUrl":"https://media.releases.sh/orgs/dagster.jpg","githubHandle":"dagster-io","description":null},{"kind":"org","slug":"clickhouse","name":"ClickHouse","domain":"clickhouse.com","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse","description":null},{"kind":"org","slug":"tinybird","name":"Tinybird","domain":"tinybird.co","avatarUrl":"https://media.releases.sh/orgs/tinybird.png","githubHandle":"tinybirdco","description":null}],"orgs":[{"slug":"dbt-labs","name":"dbt Labs","domain":"getdbt.com","avatarUrl":"https://media.releases.sh/orgs/dbt-labs.png","githubHandle":"dbt-labs","description":null},{"slug":"dagster","name":"Dagster","domain":"dagster.io","avatarUrl":"https://media.releases.sh/orgs/dagster.jpg","githubHandle":"dagster-io","description":null},{"slug":"clickhouse","name":"ClickHouse","domain":"clickhouse.com","avatarUrl":"https://media.releases.sh/orgs/clickhouse.png","githubHandle":"ClickHouse","description":null},{"slug":"tinybird","name":"Tinybird","domain":"tinybird.co","avatarUrl":"https://media.releases.sh/orgs/tinybird.png","githubHandle":"tinybirdco","description":null}],"releases":[{"id":"rel_3V2cCVVg0JwYaaXEc8i0l","version":null,"type":"feature","title":"ClickHouse Unifies Enterprise AI Data Stack with Launch of Expanded Platform","summary":"Now serving more than 5,000 customers, ClickHouse is introducing new capabilities spanning Postgres, observability and security, alongside its acquisition of Xata, challenging established vendors across the enterprise.","titleGenerated":null,"titleShort":null,"importance":null,"content":"Now serving more than 5,000 customers, ClickHouse is introducing new capabilities spanning Postgres, observability and security, alongside its acquisition of Xata, challenging established vendors across the enterprise.","publishedAt":"2026-11-09T17:01:12.000Z","url":"https://clickhouse.com/blog/clickhouse-unifies-enterprise-ai-data-stack","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_GOk4RC4ZjeWXb1ay4hLoT","version":null,"type":"feature","title":"Announcing ClickHouse Observability - fully managed observability on ClickHouse Cloud","summary":"Introducing ClickHouse Observability: fully managed observability with the speed and scale of ClickHouse. Just send telemetry; we'll handle the rest. Plus, build the observability experience you've always wanted with a simple prompt in Observability Apps.","titleGenerated":null,"titleShort":null,"importance":null,"content":"Introducing ClickHouse Observability: fully managed observability with the speed and scale of ClickHouse. Just send telemetry; we'll handle the rest. Plus, build the observability experience you've always wanted with a simple prompt in Observability Apps.","publishedAt":"2026-11-09T17:00:00.000Z","url":"https://clickhouse.com/blog/announcing-clickhouse-observability","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_XKADUXGDXp9wSlx220ckk","version":null,"type":"feature","title":"Introducing ClickHouse Security in private preview, built on RunReveal","summary":"ClickHouse Security, an agentic SIEM built on RunReveal and ClickHouse Cloud, is now in private preview, with AI agents, 100+ integrations and built-in detections running on your own ClickHouse service.","titleGenerated":null,"titleShort":null,"importance":null,"content":"ClickHouse Security, an agentic SIEM built on RunReveal and ClickHouse Cloud, is now in private preview, with AI agents, 100+ integrations and built-in detections running on your own ClickHouse service.","publishedAt":"2026-11-09T16:59:45.000Z","url":"https://clickhouse.com/blog/introducing-clickhouse-security-agentic-siem","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_QBPbXYNsqxEDz9FCpStKz","version":null,"type":"feature","title":"Langfuse launches AI Gateway","summary":"One open source entry point for all AI usage in your company. See every model call, who made it and what it cost, natively traced in Langfuse.","titleGenerated":null,"titleShort":null,"importance":null,"content":"One open source entry point for all AI usage in your company. See every model call, who made it and what it cost, natively traced in Langfuse.","publishedAt":"2026-11-09T16:57:37.000Z","url":"https://clickhouse.com/blog/langfuse-ai-gateway","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_GOUko-TsgsD5paf0s4MvT","version":null,"type":"feature","title":"ClickHouse Managed Postgres goes GA and Xata acquisition brings agentic capabilities to the service","summary":"ClickHouse Managed Postgres, our NVMe-backed Postgres service natively connected to ClickHouse, is generally available, and we're acquiring Xata to add instant branching, cloning and scale-to-zero for developers and AI agents.","titleGenerated":null,"titleShort":null,"importance":null,"content":"ClickHouse Managed Postgres, our NVMe-backed Postgres service natively connected to ClickHouse, is generally available, and we're acquiring Xata to add instant branching, cloning and scale-to-zero for developers and AI agents.","publishedAt":"2026-11-09T16:56:11.000Z","url":"https://clickhouse.com/blog/managed-postgres-ga-xata-acquisition","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_PSKZwTBmjuv8h6OQN3tFL","version":null,"type":"feature","title":"The largest expansion of ClickHouse yet","summary":"ClickHouse Managed Postgres is generally available, ClickHouse Observability enters public beta, ClickHouse Security launches in private preview, Langfuse introduces an open-source AI Gateway, and ClickHouse acquires Xata.","titleGenerated":null,"titleShort":null,"importance":null,"content":"ClickHouse Managed Postgres is generally available, ClickHouse Observability enters public beta, ClickHouse Security launches in private preview, Langfuse introduces an open-source AI Gateway, and ClickHouse acquires Xata.","publishedAt":"2026-11-09T00:45:00.000Z","url":"https://clickhouse.com/blog/announcing-clickhouse-platform-largest-expansion","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_p-zJiPUZ_Z2NJuqU4x4cB","version":null,"type":"feature","title":"What I learned from a month with WalShadow on ClickHouse Managed Postgres","summary":"Hidehiko ASAHI shares a month of WalShadow private-preview testing, comparing replication latency with ClickPipes and checking data correctness, schema changes, and recovery.","titleGenerated":null,"titleShort":null,"importance":null,"content":"Hidehiko ASAHI shares a month of WalShadow private-preview testing, comparing replication latency with ClickPipes and checking data correctness, schema changes, and recovery.","publishedAt":"2026-11-06T14:57:28.000Z","url":"https://clickhouse.com/blog/a-month-with-walshadow-clickhouse-managed-postgres","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_pqTMl-HAuwJdFj7_CVUjG","version":"26.10","type":"feature","title":"ClickHouse release 26.10","summary":"ClickHouse 26.10 introduces function composition, more flexible joins, faster GROUP BY queries, disk spilling for IN subqueries, an experimental columns cache, and Freeform and SQLite formats.","titleGenerated":null,"titleShort":null,"importance":null,"content":"ClickHouse 26.10 introduces function composition, more flexible joins, faster GROUP BY queries, disk spilling for IN subqueries, an experimental columns cache, and Freeform and SQLite formats.","publishedAt":"2026-11-06T11:45:40.000Z","url":"https://clickhouse.com/blog/clickhouse-release-26-10","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_rjNGsBh7-QUmsr-nU6LX2","version":"v1.11.17","type":"feature","title":"dbt v1.11.17","summary":"dbt deps now supplies credentials for git packages through a git credential helper instead of embedding them in the clone URL, so clones of redirecting repositories keep working with newer libcurl. Git packages installed from a subdirectory no longer fail with \"could not read Username\", and JSON schema validation now recognizes supported dbt State configurations on snapshots and data tests.","titleGenerated":"dbt-core 1.11.17 fixes dbt deps git credential handling","titleShort":"dbt deps git clones keep credentials via helper","importance":2,"content":"## dbt-core 1.11.17 - November 06, 2026\n\n### Fixes\n\n- Supply credentials for `dbt deps` git packages through a git credential helper instead of embedding them in the clone URL, so clones of redirecting repositories keep working with libcurl 7.81.0-1ubuntu1.30 and later ([#16861](https://github.com/dbt-labs/dbt-core/issues/16861))\n- Fix `dbt deps` failing with \"could not read Username\" when installing a git package from a `subdirectory` with credentials in the URL, by supplying credentials to the checkout's `git reset --hard` as well ([#16861](https://github.com/dbt-labs/dbt-core/issues/16861))\n- Recognize supported dbt State configurations on snapshots and data tests during JSON schema validation ([#N/A](https://github.com/dbt-labs/dbt-core/issues/N/A))\n\n### Contributors\n- [@ash2shukla](https://github.com/ash2shukla) ([#16861](https://github.com/dbt-labs/dbt-core/issues/16861), [#16861](https://github.com/dbt-labs/dbt-core/issues/16861))\n- [@dataders](https://github.com/dataders) ([#N/A](https://github.com/dbt-labs/dbt-core/issues/N/A))\n","publishedAt":"2026-11-06T07:07:47.000Z","url":"https://github.com/dbt-labs/dbt/releases/tag/v1.11.17","media":[],"prerelease":false,"source":{"slug":"dbt-core","name":"dbt-core","type":"github"},"org":{"slug":"dbt-labs","name":"dbt Labs"},"product":null,"groupSlug":"dbt-core","groupName":"dbt-core","coverageCount":0,"composition":{"bugs":3,"features":0,"enhancements":0}},{"id":"rel_98988EZeMFXTz61HMj79R","version":null,"type":"feature","title":"A smaller Tinybird Local image","summary":"The Tinybird Local image is about a third smaller to download and uses less memory when idle.","titleGenerated":"Tinybird Local image shrinks by a third, uses less idle memory","titleShort":"Tinybird Local image a third smaller, less idle memory","importance":2,"content":"The Tinybird Local image is about a third smaller to download and uses less memory when idle.","publishedAt":"2026-11-06T00:00:00.000Z","url":"https://www.tinybird.co/docs/changelog/2026-11-06-november-week-one","media":[],"prerelease":false,"source":{"slug":"tinybird-changelog","name":"Tinybird Changelog","type":"feed"},"org":{"slug":"tinybird","name":"Tinybird"},"product":null,"groupSlug":"tinybird-changelog","groupName":"Tinybird Changelog","coverageCount":0,"composition":{"bugs":0,"features":0,"enhancements":1}},{"id":"rel_mg2vkV60Y1GhswmWNl2XV","version":"1.13.29","type":"feature","title":"1.13.29 (core) / 0.29.29 (libraries)","summary":"The dg CLI now shows its options and commands again when typer 0.26 or later is installed. Shift-click selection in run, asset, and automation tables now works when the range crosses row 10, and selecting a large asset group no longer freezes the page.","titleGenerated":"Dagster 1.13.29 fixes dg --help output and shift-click table selection","titleShort":"dg --help restored; shift-click selection fixed past row 10","importance":2,"content":"### New\n\n- [ui] Run pages now start loading logs sooner.\n- [ui] The code location overview, deployment code locations list, and sensor and schedule detail pages now have mobile layouts.\n- [ui] Improved global search performance on the home page and catalog.\n- [ui] Improved global search responsiveness for deployments with many assets.\n- [ui] Improved asset catalog load performance by sharing asset selection filtering across the page.\n\n### Bugfixes\n\n- [dg] Fixed `dg --help` showing no options or commands when `typer` 0.26 or later was installed. (Thanks, [@DRACULA1729](https://github.com/DRACULA1729)!)\n- [ui] Fixed shift-click selection in run, asset, and automation tables selecting nothing when the range crossed row 10.\n- [ui] Selecting a large asset group in the catalog with its group checkbox no longer freezes the page.\n- [ui] Fixed asset pages opening in the desktop layout on phones.\n- [ui] Fixed the Runs page's bulk actions menu so that actions the user doesn't have permission for cannot be taken, and are shown as disabled.\n- [ui] Fixed duplicate and stale rows accumulating in global search results when editing the query.","publishedAt":"2026-11-05T18:38:24.000Z","url":"https://github.com/dagster-io/dagster/releases/tag/1.13.29","media":[],"prerelease":false,"source":{"slug":"dagster","name":"Dagster","type":"github"},"org":{"slug":"dagster","name":"Dagster"},"product":null,"groupSlug":"dagster","groupName":"Dagster","coverageCount":0,"composition":{"bugs":6,"features":0,"enhancements":5}},{"id":"rel_B5yyPdLJDUQcHMMj8qxcD","version":null,"type":"feature","title":"Fencing a Postgres primary that might still be alive","summary":"Promoting a standby doesn't stop the old Postgres primary from taking writes. Here's how the control plane behind ClickHouse Managed Postgres fences it on bare metal, AWS and GCP, and why the design is fail-open.","titleGenerated":null,"titleShort":null,"importance":null,"content":"Promoting a standby doesn't stop the old Postgres primary from taking writes. Here's how the control plane behind ClickHouse Managed Postgres fences it on bare metal, AWS and GCP, and why the design is fail-open.","publishedAt":"2026-11-05T00:00:00.000Z","url":"https://clickhouse.com/blog/fencing-a-postgres-primary-that-might-still-be-alive","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_6QMK2QVMYDWc5-Zj-2pY3","version":null,"type":"feature","title":"MongoDB CDC connector for ClickPipes is now generally available","summary":"The MongoDB CDC connector for ClickPipes is now generally available, with parallel snapshotting, structured ingestion into typed columns, lag and resource metrics, and Terraform and OpenAPI support.","titleGenerated":null,"titleShort":null,"importance":null,"content":"The MongoDB CDC connector for ClickPipes is now generally available, with parallel snapshotting, structured ingestion into typed columns, lag and resource metrics, and Terraform and OpenAPI support.","publishedAt":"2026-11-04T17:46:41.000Z","url":"https://clickhouse.com/blog/mongodb-cdc-connector-clickpipes-ga","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_sy-NlPG7vpJvm2ooINQO3","version":null,"type":"feature","title":"What's new in ClickStack - September and October '26","summary":"The September and October ClickStack releases bring ad hoc dashboard filters, PromQL for every chart type, ClickStack query tracing, a Trace logs tab, alerts from the chart explorer, and a long list of search page and collector improvements.","titleGenerated":null,"titleShort":null,"importance":null,"content":"The September and October ClickStack releases bring ad hoc dashboard filters, PromQL for every chart type, ClickStack query tracing, a Trace logs tab, alerts from the chart explorer, and a long list of search page and collector improvements.","publishedAt":"2026-11-03T19:09:52.000Z","url":"https://clickhouse.com/blog/whats-new-in-clickstack-october-2026","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_bKoGYI7NDhgCfxUGBSn9q","version":"v2.0.11","type":"feature","title":"v2.0.11","summary":"The --run-cache-mode CLI flag is now --state-mode, matching dbt State terminology, and --run-cache-mode keeps working as a deprecated alias with a warning until it is removed. dbt deps now passes git package URLs, revisions, and subdirectories after --, and a panic when the project directory is on a Windows network share is fixed.","titleGenerated":"dbt-core v2.0.11 renames --run-cache-mode to --state-mode","titleShort":"--run-cache-mode renamed to --state-mode; old flag deprecated","importance":3,"content":"## 2.0.11\n\nReleased November 03, 2026\n\n### Features\n\n- Add support for BigQuery query reservation configuration at model and connection levels ([#15486](https://github.com/dbt-labs/dbt-core/issues/15486))\n- Report whether each node ran remotely, locally, or on Lake Compute in anonymous usage telemetry.\n- Show a determinate progress bar for the parse phase in interactive terminals, matching the render/analyze/run experience, and name the stage in progress (building model nodes, resolving data tests, typechecking macros) during the stretches where no assets are being parsed, plus a progress indicator while the semantic manifest is validated, so the console no longer looks idle\n- Rename the --run-cache-mode CLI flag to --state-mode to match the dbt State terminology it configures. --run-cache-mode is kept as a deprecated alias and will keep working, with a warning, until it is removed in a future release.\n- Add state configuration to ignore external modifications on model target tables/views (only implemented for views to begin with)\n- Support the `allow_jinja_file_extensions` project flag, so resource files may carry a trailing Jinja suffix (`model.sql.j2`, `doc.md.j2`) ([#16296](https://github.com/dbt-labs/dbt-core/issues/16296))\n- Add telemetry for state selector operations to track performance and usage of modified/new/unmodified node selection\n- Support authenticated private S3 downloads for dbt update manifests and binaries.\n- Add DBT_CLOUD_LOGIN_HOST (and optional DBT_CLOUD_LOGIN_ACCOUNT_ID) so dbt login can authenticate through an account host's /oauth/authorize instead of the registration page, for SSO accounts the registration page cannot route.\n- [docs v2] Add resource filter to lineage view to select what resource types are displayed in the DAG\n- Add a feature flag registry with `--feature NAME[=VALUE]` and `DBT_ENGINE_FEATURES` for setting unstable features; behavior flags and the git deps fast path now resolve through it with unchanged defaults\n- Add the ThreadPoolWait telemetry span for runtime pool queue waits, superseding ConnectionLimitWait\n- Report individual microbatch executions as MicrobatchEvaluated spans in telemetry, file logs, and Core-compatible JSON logs, and list them under their model in interactive output\n- Thread pool wait spans now record the unique_id and batch_id of the node or microbatch they belong to\n- Read and publish AWS Databricks Unity Catalog tables through Lake Compute using profile PAT or OAuth M2M credentials. Use canonical lower-case relation names and full-refresh existing models when adopting propagation with mixed-case database or schema names.\n\n### Fixes\n\n- Fix modules.re functions silently dropping keyword arguments (flags=, maxsplit=, count=) ([#15508](https://github.com/dbt-labs/dbt-core/issues/15508))\n- Emit only changed columns in Databricks column-tag alter diffs ([#16246](https://github.com/dbt-labs/dbt-core/issues/16246))\n- Emit only changed keys in Databricks relation-tag SET TAGS diffs ([#16248](https://github.com/dbt-labs/dbt-core/issues/16248))\n- Emit only changed Databricks column-tag keys in ALTER COLUMN SET TAGS diffs ([#16272](https://github.com/dbt-labs/dbt-core/issues/16272))\n- Implement TableRepr methods index_of_value_in_column and count_occurrences_of_value_in_column to fix Fusion panic in dbt_utils.get_query_results_as_dict()\n- Copy config.group onto the top-level group for source and singular tests, matching dbt-core ([#15930](https://github.com/dbt-labs/dbt-core/issues/15930))\n- Snowflake interactive_table: correct the dbt1005 message for an invalid cluster_by — every entry must be non-blank, not just one\n- Make insert_gateway_label idempotent so gateway host resolution no-ops when the label is already present, instead of relying on a caller-side guard\n- Set tags on operation nodes so on-run-start/on-run-end hooks are discoverable when running dbt-core with --use-v2-parser ([#16310](https://github.com/dbt-labs/dbt-core/issues/16310))\n- Allow integer `attribute` in the `sort`, `join`, `unique`, and `groupby` Jinja filters (e.g. `sort(attribute=0)`), matching Jinja2 behavior\n- Match BigQuery snapshot check columns without regard to case to avoid false row changes.\n- Keep the existing Databricks table when a view model that replaces it fails to build without use_materialization_v2 ([#16476](https://github.com/dbt-labs/dbt-core/issues/16476))\n- A unit test that overrides one package macro (my_package.my_macro in overrides.macros) no longer makes the package's other macros undefined, matching dbt-core ([#16477](https://github.com/dbt-labs/dbt-core/issues/16477))\n- Support `exposure` in the list command's `--resource-type` option ([#16224](https://github.com/dbt-labs/dbt-core/issues/16224))\n- Require matching existing and target relation types for Databricks view_update_via_alter updates ([#16538](https://github.com/dbt-labs/dbt-core/issues/16538))\n- Fix a panic when the project directory is on a Windows network share (UNC path or mapped network drive) ([#16428](https://github.com/dbt-labs/dbt-core/issues/16428))\n- Unit tests with `overrides.vars` no longer drop project vars; overrides layer over `--vars` like dbt Core\n- `var.has_var` now also reports vars passed with `--vars`, matching dbt Core\n- Sidecar: Fixed interpolated values for getting relation info\n- docs-v2 source detail page showed the first table's description instead of the source's own\n- Refresh an account-scoped OAuth session when a server rejects an access token before its cached expiry.\n- Validate refreshed OAuth credentials against the cached account and host before persisting.\n- Preserve Snowflake nested JSON fixture escapes and heterogeneous array values during local execution.\n- Commit a rebuilt metadata index only once every table has been written, so a write that fails partway through leaves the previous index in place instead of a mix of new and stale files.\n- Add copy_grants support to Snowflake dynamic tables\n- Find user-installed dbt executables and Windows virtual environments reliably.\n- Fix the `Downloading relations` progress bar rendering as a stuck 0/N bar. Adapters with no per-item progress reporting now render a spinner instead; Snowflake now reports real per-schema progress so its bar fills as expected\n- Replace `DBT_AI_PROVIDER` with `DBT_ENGINE_AI_PROVIDER` ([#16312](https://github.com/dbt-labs/dbt-core/issues/16312))\n- The environment variables behind `--generate-info-schema` and `--info-schema-dir` are now `DBT_ENGINE_GENERATE_INFO_SCHEMA` and `DBT_ENGINE_INFO_SCHEMA_DIR`. They were named `DBT_GENERATE_INFO_SCHEMA` and `DBT_INFO_SCHEMA_DIR`, outside the reserved `DBT_ENGINE_` prefix that the engine's other variables use. The flags themselves are unchanged.\n- Setting `DBT_ENGINE_AI_PROVIDER` no longer aborts the invocation. The variable was moved onto the reserved `DBT_ENGINE_` prefix without being registered as a known engine variable, so `validate_engine_env_vars` rejected it as user-authored and every command that reached setup exited 1 with dbt1005.\n- Support credential-independent profile metadata and scoped environment values while preserving secret access rules.\n- Match BigQuery partition fields case-insensitively when inspecting existing tables ([#14646](https://github.com/dbt-labs/dbt-core/issues/14646))\n- Accept compute_region and legacy dataproc_region together in BigQuery profiles.\n- Populate complete query tags for Databricks seeds; fix materialization type for seeds to be `seed` rather than `table` ([#15931](https://github.com/dbt-labs/dbt-core/issues/15931))\n- BigQuery unit tests no longer leak partition pseudocolumns into expected results ([#15357](https://github.com/dbt-labs/dbt-core/issues/15357))\n- Fix broken image links in documentation ([#16288](https://github.com/dbt-labs/dbt-core/issues/16288))\n- Protect Windows process environment variables from project .env overrides.\n- Replace deprecated DuckDB single-arrow lambdas with Python-style lambda syntax.\n- Reuse cached relation metadata when checking stale views during development cloning.\n- Honor valid dbt Core-generated package lock checksums for common package declarations with best-effort compatibility.\n- Correct how dbt State classifies model materializations: materialized views submit as VIEW, adapter-specific materializations such as dynamic tables and metric views submit as DBT_CUSTOM (and are therefore no longer eligible as clone candidates), semantic views are no longer submitted, and full-refreshed snapshots submit as FULL\n- Databricks: include all models in catalog.json for schemas with more than 50 relations ([#15795](https://github.com/dbt-labs/dbt-core/issues/15795))\n- Report the effective dbt State defer-to target, honoring a target's `defer_to_target`, instead of always reporting the default.\n- Order a unit test's render after the seed that materializes an overlapping source, so dbt build no longer reads the relation before the seed has created it. This ordering now also holds when the tested model is not selected, for example when only the seed and the unit test are selected\n- Preserve BigQuery GEOGRAPHY, TIMESTAMP, and JSON types in dbt OSS unit test fixtures ([#16410](https://github.com/dbt-labs/dbt-core/issues/16410))\n- Exempt ad-hoc inline (preview) nodes from the model access check, so `dbt show --inline` and `dbt compile --inline` can `ref()` a private or protected model\n- Fix Redshift OAuth authentication failures with an updated ADBC driver.\n- Add new STRUCT and ARRAY columns to BigQuery snapshot tables with their SQL types instead of `RECORD` or the array's element type ([#16350](https://github.com/dbt-labs/dbt-core/issues/16350))\n- Keep the microbatch event-time filter on relations derived from a ref with include(), quote(), replace_path(), incorporate() or without_identifier() ([#16383](https://github.com/dbt-labs/dbt-core/issues/16383))\n- Refresh persisted unit-test given fixture schemas so warehouse DDL changes are reflected.\n- Send Redshift's case-sensitive sqlglot dialect setting to the server when enable_case_sensitive_identifier is enabled\n- Fix DuckDB seeds defined in packages failing to load because the CSV path duplicated the package directory.\n- Add duckdb__date dispatch macro so dbt.date() / dbt.date_spine() work on DuckDB targets\n- Retry transient Redshift query-execution errors (catalog races, serialization conflicts) on the same connection\n- Warn on missing persist_docs columns for Databricks table, incremental, materialized view, and streaming table; fix V2 column comments being gated on the wrong persist_docs config key\n- Snowflake incremental models on catalog-linked databases (e.g. Glue) no longer fail on `DROP TABLE ... CASCADE` of the `__dbt_tmp` relation\n- Time Machine replay: rebuild the relations of a recorded `list_relations_in_parallel` result instead of replaying every listed schema as empty, so `--defer` no longer treats nodes present in the dev schema as absent and re-points their refs at the deferred relation\n- Time Machine replay now preserves Snowflake column precision/scale (e.g. NUMBER(38,6), TIMESTAMP_LTZ(6)) so unit-test fixture SQL matches the recording\n- Finalize invocation telemetry when execution ends even if background workers retain descendant spans.\n- Fix DuckDB and Databricks incremental/snapshot builds panicking with an unhandled Arrow BinaryView type on BLOB/binary columns ([#16364](https://github.com/dbt-labs/dbt-core/issues/16364))\n- Include the underlying cause in package hub and tarball request errors, and stop reporting a 500 status when no response was received\n- Auto-exposures discovered from BI tools now carry the target's adapter instead of defaulting to Snowflake, so non-Snowflake projects no longer fail selection with dbt1005 'selected nodes need adapters the target does not declare'\n- Stop emitting the `with connection` clause for BigQuery Iceberg tables using `lakehouse_catalog` (LRC) when `use_catalogs_v2` is left at its default.\n- Inline SQL and CSV unit-test fixture file contents into the parsed manifest and resolve fixtures in dependency packages. ([#16354](https://github.com/dbt-labs/dbt-core/issues/16354))\n- Respect project-level overrides of create_schema and other adapter macros invoked from Rust. ([#16346](https://github.com/dbt-labs/dbt-core/issues/16346))\n- Avoid unsafe CREATE OR REPLACE TABLE operations in BigQuery incremental and snapshot materializations. ([#15953](https://github.com/dbt-labs/dbt-core/issues/15953))\n- Fix BigQuery incremental models being recreated instead of merged when a concurrent table drop makes a dataset listing fail\n- Skip per-table metadata when hydrating the BigQuery relation cache through ADBC GetObjects. ([#16425](https://github.com/dbt-labs/dbt-core/issues/16425))\n- BigQuery freshness checks are now faster and safer. Freshness scans for different schemas now run at the same time. Freshness checks for the same schema and different nodes now share one request.\n- Strip CR and BOM from PEM-headered Snowflake private keys so keys saved with Windows line endings decode correctly ([#15107](https://github.com/dbt-labs/dbt-core/issues/15107))\n- Snowflake seeds now honor the `transient` config instead of silently creating a permanent table\n- Fetch the tested model's schema for incremental unit tests with static analysis enabled instead of failing with dbt9002\n- Process explainer GetExplainMessage requests in batches.\n- Preserve BigQuery decimal and nested types in unit test fixtures ([#16440](https://github.com/dbt-labs/dbt-core/issues/16440))\n- Correct the `dbt show --info` help text, which still said a prior `--generate-info-schema` run was required\n- DuckDB unit test fixtures for TIMESTAMP WITH TIME ZONE columns are no longer cast to a naive timestamp, which previously dropped the offset and shifted the value by the session timezone ([#16470](https://github.com/dbt-labs/dbt-core/issues/16470))\n- Fix dbt-dist recursion on version manager shims ([#16537](https://github.com/dbt-labs/dbt-core/issues/16537))\n- Fix `--write-index` and `--dirty` runs missing files added since the last parse: new models, seeds, macros and other resources are now loaded, `--dirty` selects newly added SQL models and analyses, and `--no-partial-load` now turns off the cached-state shortcut.\n- Honor OS-assigned loopback ports in OAuth authorization and token-exchange callback URLs.\n- Recognize `$$...$$` dollar-quoted strings in the Databricks SQL lexer so a `;` inside a metric view's YAML body (e.g. in a `comment:` block) no longer truncates the compiled `create or replace view` DDL. ([#16385](https://github.com/dbt-labs/dbt-core/issues/16385))\n- Return an empty column list when Databricks get_columns_in_relation targets a missing hive_metastore relation ([#16472](https://github.com/dbt-labs/dbt-core/issues/16472))\n- Clarify the warning for a package-lock.yml written by an older dbt version and point to `dbt deps`\n- Removed lingering 'dbt' naming from parse-failure and warn_error_options log output, and pointed the parse-failure suggestion at the dbt v2 upgrade guide.\n- Bring Databricks macros in line with dbt-databricks v1.12.4\n- Fix nil pointer dereference panic in BigQuery foundry driver\n- Add ability to record seeds as transient table types for Snowflake\n- Convert Arrow FixedSizeList values to Jinja so Snowflake VECTOR columns do not panic when read into an agate table.\n- Databricks Python models: wait for the all-purpose cluster to be running before creating the execution context, surface Command API error messages, and stop warning on an unset timeout\n- Bump lake compute driver to enable jwt authentication users to no longer need a background pat to be generated.\n- Repeating a single-value flag such as `--target` or `--debug` keeps the last value instead of failing with a usage error, matching dbt Core.\n- Send allow_clones, is_defer_to_profile, and defer_enabled on dbt State submit requests for explainer parity\n- [Redshift] Fix issues with external OAuth breaking for multi-threaded execution ([#15555](https://github.com/dbt-labs/dbt-core/issues/15555))\n- Write `warn_error_options` to `run_results.json` args, matching dbt Core.\n- Databricks: views and metric views under an iceberg catalog no longer fail with 'materialized must be incremental, table, or seed' when catalogs.yml v2 is enabled. ([#16474](https://github.com/dbt-labs/dbt-core/issues/16474))\n- Stop panicking on Jinja AgateTable comparisons; compare tables by identity like dbt 1.x ([#15920](https://github.com/dbt-labs/dbt-core/issues/15920))\n- Fix `model.resource_type` returning model for all resource types during parse ([#16637](https://github.com/dbt-labs/dbt-core/issues/16637))\n- Build source relations the same way as `source()`, so they skip the adapter's max identifier length check (matching dbt Core) and use `catalog_name` as the database\n- Fix BigQuery table models being dropped before every rebuild when a cluster_by entry is backtick-quoted or differs in letter case from the stored column\n- Restore connection wait telemetry for runtime pool contention on task operations.\n- Include semantic_manifest.json in the artifact ingest upload when the run writes it\n- A microbatch model's idle time and duration no longer overcount waits of its concurrent batches; the model is idle only while none of its batches runs\n- After a concurrent microbatch batch fails, batches not yet started are canceled instead of running after their model has failed\n- Verify TLS certificates against the operating system's certificate store instead of a bundled CA list, so HTTPS connections work behind TLS-inspecting proxies whose CA is only installed in the OS store.\n- Always mark reused passing dbt State tests as reused in run results.\n\n### Under the Hood\n\n- Refresh README with a dbt v2.0 DAG screenshot stored in `assets/` and point the CI badge at the current `dbt-labs/dbt` workflow.\n- Allow metadata freshness checks to use an explicit artifact directory.\n- Allow base Jinja context builders to reuse a caller-provided ResultStore.\n- Log a warning naming the relation when no schema is returned for a dangling source during static analysis hydration, instead of skipping it silently.\n- Reuse the parsed Jinja AST when extracting unrendered config() arguments.\n- Extend package metadata with cross-project environment routing fields.\n- dbt-parser: avoid a second, redundant Jinja parse when statically recovering source() calls hidden in a conditional branch during model, function, and singular data test rendering\n- Add a per-resource-type applicability table for adapter-specific config keys, replacing the shared WarehouseSpecificNodeConfig field list with per-(resource type, key) status. A key set on a resource type that does not support it now emits a DeprecatedConfigKey warning. On schema.yml and inline config() the key is still applied, so behavior is unchanged; in dbt_project.yml, where the key has no field to deserialize into and already warned, it is dropped.\n- Time Machine replay pins the Jinja wall clock to the recording start, so wall-clock-derived SQL (e.g. a `utcnow()`-based var) matches the recorded text\n- Add force closing of span subtrees to dbt-tracing, ending a span's structured telemetry even while other handles to it or its descendants are alive.\n- docs-v2: document the __overview__ landing-page customization block in the README\n- Ignore `flags:` in installed packages' `dbt_project.yml`: project flags, including behavior change and adapter behavior flags, only apply from the root project\n\n### Dependencies\n\n- Update ADBC drivers to Snowflake 0.21.21, BigQuery 0.21.23, BigQuery Foundry 0.1.11, Databricks 0.21.15, Redshift 0.18.11, Spark 0.1.4, DuckDB Extended 0.0.32, Lake Compute 0.1.13, and MSSQL 1.7.1 (dbt build suffixes).\n\n### Security\n\n- Pass git package URLs, revisions, and subdirectories to git after `--` in `dbt deps`\n- Harden Databricks logged-SQL credential redaction: match `credential` and `encryption` clauses case-insensitively, redact every clause rather than only the first, and correctly handle option values containing commas, newlines, quotes or escaped quotes and keys containing dots. ([#16053](https://github.com/dbt-labs/dbt-core/issues/16053))\n- Checks and `dbt show --info`/`--inline` now run in a DuckDB database that can read only the project metadata: no other files, no network access, and no extension installs\n- Upgrade gix to 0.84.0 to address security advisories in the gix dependency family.\n- Update BigQuery and Spark ADBC driver builds to Go 1.26.10.\n- Update rustls-webpki to 0.103.16 and webpki-root-certs to 1.0.10 to reject distrusted certificate chains.\n\n### Contributors\n- [@HannanNaeem](https://github.com/HannanNaeem) ([#16296](https://github.com/dbt-labs/dbt-core/issues/16296), [#16428](https://github.com/dbt-labs/dbt-core/issues/16428))\n- [@Jules-G1](https://github.com/Jules-G1)\n- [@Oliver-Weng-Sorcerer](https://github.com/Oliver-Weng-Sorcerer)\n- [@ShaneMcGovern](https://github.com/ShaneMcGovern) ([#16224](https://github.com/dbt-labs/dbt-core/issues/16224))\n- [@TIHan](https://github.com/TIHan)\n- [@VaggelisD](https://github.com/VaggelisD)\n- [@aahel](https://github.com/aahel)\n- [@aiguofer](https://github.com/aiguofer) ([#16310](https://github.com/dbt-labs/dbt-core/issues/16310))\n- [@ajhlee-dbt](https://github.com/ajhlee-dbt) ([#16288](https://github.com/dbt-labs/dbt-core/issues/16288), [#15555](https://github.com/dbt-labs/dbt-core/issues/15555), [#16637](https://github.com/dbt-labs/dbt-core/issues/16637))\n- [@akbog](https://github.com/akbog)\n- [@alex-mirkin](https://github.com/alex-mirkin) ([#16383](https://github.com/dbt-labs/dbt-core/issues/16383))\n- [@ayshukla](https://github.com/ayshukla)\n- [@b-per](https://github.com/b-per) ([#16364](https://github.com/dbt-labs/dbt-core/issues/16364))\n- [@chayac](https://github.com/chayac)\n- [@claude](https://github.com/claude)\n- [@dataders](https://github.com/dataders)\n- [@dbeatty10](https://github.com/dbeatty10) ([#16312](https://github.com/dbt-labs/dbt-core/issues/16312))\n- [@debrin-og](https://github.com/debrin-og) ([#15931](https://github.com/dbt-labs/dbt-core/issues/15931), [#15795](https://github.com/dbt-labs/dbt-core/issues/15795), [#15920](https://github.com/dbt-labs/dbt-core/issues/15920))\n- [@eakmanrq](https://github.com/eakmanrq)\n- [@eliasdefaria](https://github.com/eliasdefaria)\n- [@ernestoongaro](https://github.com/ernestoongaro)\n- [@felipecrv](https://github.com/felipecrv)\n- [@fivetran-bradbeggs](https://github.com/fivetran-bradbeggs)\n- [@fivetran-chadboyd](https://github.com/fivetran-chadboyd)\n- [@fivetran-vladimircvetanovic](https://github.com/fivetran-vladimircvetanovic)\n- [@itsannbat](https://github.com/itsannbat)\n- [@itsnamangoyal](https://github.com/itsnamangoyal)\n- [@izeigerman](https://github.com/izeigerman)\n- [@j-clemons](https://github.com/j-clemons) ([#16537](https://github.com/dbt-labs/dbt-core/issues/16537))\n- [@j19s84](https://github.com/j19s84)\n- [@jasonlin45](https://github.com/jasonlin45)\n- [@jeremyyeo](https://github.com/jeremyyeo)\n- [@jess-scott](https://github.com/jess-scott)\n- [@joaodaher](https://github.com/joaodaher)\n- [@kczimm](https://github.com/kczimm)\n- [@larspettermadsstuen](https://github.com/larspettermadsstuen) ([#15930](https://github.com/dbt-labs/dbt-core/issues/15930))\n- [@lottaquestions](https://github.com/lottaquestions) ([#16354](https://github.com/dbt-labs/dbt-core/issues/16354), [#16346](https://github.com/dbt-labs/dbt-core/issues/16346), [#16470](https://github.com/dbt-labs/dbt-core/issues/16470))\n- [@luc-pimentel](https://github.com/luc-pimentel) ([#16477](https://github.com/dbt-labs/dbt-core/issues/16477))\n- [@mach-kernel](https://github.com/mach-kernel) ([#16410](https://github.com/dbt-labs/dbt-core/issues/16410), [#16440](https://github.com/dbt-labs/dbt-core/issues/16440))\n- [@max-ostapenko](https://github.com/max-ostapenko) ([#15486](https://github.com/dbt-labs/dbt-core/issues/15486))\n- [@mishamsk](https://github.com/mishamsk)\n- [@r-jais](https://github.com/r-jais) ([#15107](https://github.com/dbt-labs/dbt-core/issues/15107), [#16385](https://github.com/dbt-labs/dbt-core/issues/16385))\n- [@ragesh-g](https://github.com/ragesh-g) ([#14646](https://github.com/dbt-labs/dbt-core/issues/14646), [#16350](https://github.com/dbt-labs/dbt-core/issues/16350), [#16472](https://github.com/dbt-labs/dbt-core/issues/16472))\n- [@sadboy](https://github.com/sadboy)\n- [@sd-db](https://github.com/sd-db) ([#16246](https://github.com/dbt-labs/dbt-core/issues/16246), [#16248](https://github.com/dbt-labs/dbt-core/issues/16248), [#16272](https://github.com/dbt-labs/dbt-core/issues/16272), [#16476](https://github.com/dbt-labs/dbt-core/issues/16476), [#16538](https://github.com/dbt-labs/dbt-core/issues/16538))\n- [@serramatutu](https://github.com/serramatutu) ([#15357](https://github.com/dbt-labs/dbt-core/issues/15357))\n- [@srchilukoori](https://github.com/srchilukoori) ([#15508](https://github.com/dbt-labs/dbt-core/issues/15508))\n- [@toriwei](https://github.com/toriwei)\n- [@vchan](https://github.com/vchan)\n- [@versusfacit](https://github.com/versusfacit)\n- [@vrongmeal](https://github.com/vrongmeal) ([#16053](https://github.com/dbt-labs/dbt-core/issues/16053))\n- [@will-sargent-dbtlabs](https://github.com/will-sargent-dbtlabs) ([#16474](https://github.com/dbt-labs/dbt-core/issues/16474))\n- [@xuliangs](https://github.com/xuliangs) ([#15953](https://github.com/dbt-labs/dbt-core/issues/15953), [#16425](https://github.com/dbt-labs/dbt-core/issues/16425))\n","publishedAt":"2026-11-03T09:23:42.000Z","url":"https://github.com/dbt-labs/dbt/releases/tag/v2.0.11","media":[],"prerelease":false,"source":{"slug":"dbt-core","name":"dbt-core","type":"github"},"org":{"slug":"dbt-labs","name":"dbt Labs"},"product":null,"groupSlug":"dbt-core","groupName":"dbt-core","coverageCount":0,"composition":{"bugs":100,"features":15,"enhancements":7}},{"id":"rel_R5EY_GpzO5u07a7MtS_MN","version":"0.5.0","type":"feature","title":"What's new in clickhousectl v0.5.0","summary":"clickhousectl v0.5.0 takes UDFs from local to Cloud, adds ClickHouse Managed Postgres observability, standardises the command surface, and lists the API-key permissions every Cloud command needs.","titleGenerated":null,"titleShort":null,"importance":null,"content":"clickhousectl v0.5.0 takes UDFs from local to Cloud, adds ClickHouse Managed Postgres observability, standardises the command surface, and lists the API-key permissions every Cloud command needs.","publishedAt":"2026-11-02T14:45:00.000Z","url":"https://clickhouse.com/blog/clickhousectl-v0-5-0","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_OXK9cb4KPtBRdkzeAjCC6","version":"1.13.28","type":"feature","title":"1.13.28 (core) / 0.29.28 (libraries)","summary":"Dagster now supports protobuf 7, and dagster-cloud agents can retry code locations that fail to start via the opt-in code_location_start_retry setting. Also added a preview run_job() method to PipesDatabricksClient, a dagster-dbt[dbt] extra for dbt v2, and a UPathIOManager.storage_options fix for universal-pathlib 0.3.0 or newer.","titleGenerated":"Dagster 1.13.28 adds protobuf 7 support and code location start retries","titleShort":"Protobuf 7 supported; agents retry failed code locations","importance":2,"content":"### New\n\n- Dagster now supports protobuf 7.\n- [ui] Selection input suggestions now update when their values finish loading.\n- [ui] Selection inputs now suggest a tag key's values right after `=` or an opening quote, and while typing a tag value.\n- [ui] When no suggestion matches, selection inputs now offer the typed value in quotes if it needs them.\n- [ui] The Runs filter now lists tag keys, tag values, backfills, and partitions alphabetically.\n- [ui] Background browser tabs no longer poll for updates on job tiles, sync pages, and log views, and finished run groups refresh less often.\n- [dagster-cloud] Agents can now retry code locations that fail to start (for example, from an image pull failure or startup timeout) using the new opt-in `code_location_start_retry` setting (Helm: `workspace.codeLocationStartRetry`).\n- [dagster-databricks] Added a preview `run_job()` method to `PipesDatabricksClient` and `PipesDatabricksServerlessClient`, which triggers an existing Databricks job by ID using Dagster Pipes.\n- [dagster-databricks] Raised the `databricks-sdk` upper bound to `<0.156.0`, so newer Jobs API options like `performance_target` can be passed through `submit_args`.\n- [dagster-dbt] Added a `dagster-dbt[dbt]` extra that installs dbt v2 (the Fusion engine) from PyPI. Installing `dbt` 2.x alongside `dbt-core` now raises a clear error instead of an opaque `ImportError`.\n- [dagster-dbt] `DbtProject`, `DagsterDbtProjectPreparer`, and the `DbtProjectComponent` `project` arguments now accept `install_deps=False` to skip running `dbt deps` during project preparation, for users who install dbt packages themselves.\n- [dagster-k8s] `dagster-k8s` now supports version 37 of the `kubernetes` client.\n- [dagster-snowflake] `dagster-snowflake` now requires `snowflake-snowpark-python>=1.37.0` and `snowflake-connector-python>=3.14.0`. Older snowpark releases fail to import with setuptools 82 or newer.\n\n### Bugfixes\n\n- Fixed an issue where `UPathIOManager.storage_options`, used by IO managers such as `PolarsDeltaIOManager`, raised an `AttributeError` with universal-pathlib 0.3.0 or newer. (Thanks, [@abhishekacharya200](https://github.com/abhishekacharya200)!)\n- [dagster-dbt] Fixed a crash when using `select` or `exclude` on a dbt project with `on-run-start` or `on-run-end` hooks.\n- [dagster-dbt] Fixed a `ModuleNotFoundError: No module named 'dateutil'` when importing `dagster-dbt` without `dbt-core` installed, such as with dbt v2 (Fusion).\n\n### Dagster Plus\n\n- The audit log now records organization settings changes, removing users, team and custom role changes, API (SCIM) token creation and revocation, enabling or disabling SCIM, deployment agent type changes, and group and user changes made through SCIM provisioning.\n- The \"Runs in progress\" and \"Runs queued\" Insights metrics now capture short-lived spikes, such as bursts of very short runs, that were previously missed. Alerts on these metrics may now trigger on short bursts. Requests for these metrics that don't specify an aggregation now use Max.\n- The \"Run concurrency\" Insights metric is deprecated in favor of \"Runs in progress\".\n- The Serverless base and user-code images now ship pip 26.2.2, so `pip install` in images built on `dagster-cloud-serverless-base` no longer trusts the compromised eMudhra/emSign CA roots.\n- Fixed `add_historical_run` (used by Airlift) failing with a GraphQL error.\n- [ui] The audit log event details dialog now scales with the window, wraps long lines, and supports folding the JSON.\n- [ui] Fixed a false warning that a code location's component repository differs from the repository it deploys from.","publishedAt":"2026-10-30T15:31:35.000Z","url":"https://github.com/dagster-io/dagster/releases/tag/1.13.28","media":[],"prerelease":false,"source":{"slug":"dagster","name":"Dagster","type":"github"},"org":{"slug":"dagster","name":"Dagster"},"product":null,"groupSlug":"dagster","groupName":"Dagster","coverageCount":0,"composition":{"bugs":4,"features":8,"enhancements":8}},{"id":"rel_qjRx_P_Key43iGHtkPQvE","version":null,"type":"feature","title":"Query running merges with system.merges","summary":"Workspaces can query system.merges to list running merges and mutations with table, elapsed, progress, num_parts, and is_mutation, and system.columns no longer returns native ClickHouse columns that the replacements reject. Fixed queued imports using outdated JSONPaths after a Data Source redeploy that made S3 backfills fail with Schema mismatch, plus tb push dropping a Token's row-level filter and Developer Organization copy jobs timing out after 30 seconds.","titleGenerated":"Tinybird adds system.merges query to inspect running merges in a Workspace","titleShort":"system.merges now queryable; import JSONPath fix","importance":3,"content":"# Query running merges with system.merges¶\n\nA long merge can hold up a deployment, and until now you couldn't see which one. You can now query `system.merges` from a Workspace to list the merges and mutations running on your Data Sources:\n\n```\nSELECT table, elapsed, progress, num_parts, is_mutation\nFROM system.merges\n```\n\nThe table shows only your Workspace's database and a limited set of columns, including `table`, `elapsed`, `progress`, `num_parts`, `is_mutation`, `rows_read` and `merge_type`.\n\n`system.columns` also stopped returning native ClickHouse® columns that the system table replacements reject, so tools that build queries from it no longer get a 400.\n\n## Clearer Cluster page and Overview¶\n\nThe replicas table on the Organization Cluster page now has a header for every column, roomier Load, CPU and Memory bars, and long Cluster names no longer run into the next column. On the Overview, Cluster health now shows the same Cluster names as the Cluster page instead of internal IDs.\n\n## Improvements and bug fixes¶\n\n- \\[All\\]: Added a random delay to the `Retry-After` header on rate-limited API requests, so clients that wait and retry no longer hit the limit again at the same second.\n- \\[All\\]: Changed the Tinybird MCP Server to return tool errors with `isError: true`, so MCP clients and agent frameworks run their error handling and retries. The row limit note now appears only when a result reaches the limit.\n- \\[All\\]: Fixed queued import jobs using outdated JSONPaths after you redeploy a Data Source, which made S3 backfills fail with `Schema mismatch`. The error now lists the columns that don't match.\n- \\[All\\]: Changed role updates to reject demoting the only admin of a Workspace, so a Workspace always keeps an admin.\n- \\[All\\]: Fixed on-demand copy jobs in Developer Organizations timing out after 30 s. They now get the documented 30 minute limit.\n- \\[All\\]: Fixed the Workspaces list in the Add member dialog not scrolling with the mouse wheel or trackpad.\n- \\[All\\]: Changed the connectors API to return a 400 instead of a 500 when a request body isn't valid JSON.\n- \\[Classic\\]: Fixed `tb push` removing the row-level filter from an existing Token's read scopes when the Token was updated by name.\n- \\[CLI\\]: Improved the `tb migrate-to-forward` error when a live Classic Pipe fails to parse. It now names the Pipe instead of showing only `Empty query`.","publishedAt":"2026-10-30T00:00:00.000Z","url":"https://www.tinybird.co/docs/changelog/2026-10-30-october-week-five","media":[],"prerelease":false,"source":{"slug":"tinybird-changelog","name":"Tinybird Changelog","type":"feed"},"org":{"slug":"tinybird","name":"Tinybird"},"product":null,"groupSlug":"tinybird-changelog","groupName":"Tinybird Changelog","coverageCount":0,"composition":{"bugs":6,"features":1,"enhancements":4}},{"id":"rel_y7PpR-tDTsEJVsdrX4oGq","version":null,"type":"feature","title":"Meet GroeneAI: our autonomous contributor to ClickHouse","summary":"Meet GroeneAI, ClickHouse’s autonomous engineering agent that contributed more than 3,000 merged pull requests in under seven months.","titleGenerated":null,"titleShort":null,"importance":null,"content":"Meet GroeneAI, ClickHouse’s autonomous engineering agent that contributed more than 3,000 merged pull requests in under seven months.","publishedAt":"2026-10-29T12:53:19.000Z","url":"https://clickhouse.com/blog/meet-groeneai-our-autonomous-contributor-to-clickhouse","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null},{"id":"rel_zkck4kz3rGD2WP1VoZukl","version":null,"type":"feature","title":"The new cost-based join optimizer with reordering support for non-inner joins","summary":"ClickHouse’s new exhaustive cost-based join optimizer safely reorders inner, outer, semi, and anti joins to find the cheapest valid plan.","titleGenerated":null,"titleShort":null,"importance":null,"content":"ClickHouse’s new exhaustive cost-based join optimizer safely reorders inner, outer, semi, and anti joins to find the cheapest valid plan.","publishedAt":"2026-10-28T14:12:53.000Z","url":"https://clickhouse.com/blog/the-new-cost-based-join-optimizer-with-reordering-support-for-non-inner-joins","media":[],"prerelease":false,"source":{"slug":"clickhouse-blog","name":"ClickHouse Blog","type":"feed"},"org":{"slug":"clickhouse","name":"ClickHouse"},"product":null,"groupSlug":"clickhouse-blog","groupName":"ClickHouse Blog","coverageCount":0,"composition":null}],"pagination":{"nextCursor":"2026-10-28T14:12:53.000Z|2026-10-28T15:12:47.699Z|rel_zkck4kz3rGD2WP1VoZukl","limit":20}}