{"collection":{"slug":"data-engineering","name":"Data & Analytics Engineering"},"digest":{"id":"cwd_CDpekVa8bXBDlnUvodvZk","weekStart":"2026-07-27","title":"Automation takes the lead — jobs, ingestion, and scale","intro":"Dagster brings declarative automation to entire jobs, Tinybird finalizes its v1 local file ingestion, and ClickHouse publishes a trio of benchmarks and guides covering observability, Postgres memory management, and an espresso machine instrumented with OpenTelemetry.","body":"### Declarative automation reaches Dagster jobs\nDagster [1.13.16](/release/rel_ehffs5qZ20ftAbF9XIQ_j-1-13-16-core-0-29-16-libraries) extends its Declarative Automation system from individual assets to full jobs — a preview feature that closes a long-standing gap. You can now pass an `automation_condition` to `define_asset_job`, wrapping asset-level conditions with `AutomationCondition.any_job_root_assets_match` or `AutomationCondition.all_job_root_assets_match`. When the condition turns true, Dagster launches a single run of the entire job, and the evaluation history is visible in a new Automation tab on job pages. This is the kind of change that quietly reorients how teams think about scheduling: instead of daisy-chaining sensors and schedules, you write a condition once and let Dagster figure out the timing.\n\n### Tinybird streamlines local ingestion\nTinybird's [v1 local file ingestion](/release/rel_13Na5IoyOdsWqwUeHwhLM-v1-local-file-ingestion-is-now-the-default) is now the default on both Forward and Classic CLIs, removing the `--experimental=use_v1` flag that was required for CSV, NDJSON, or Parquet file uploads. The CLI release also tweaks job-return behavior: Forward CLI now returns the queued job ID by default (use `--wait` to block), while Classic CLI waits for completion by default (use `--no-wait` to background). File ingestion is a gateway workflow — every new pipeline starts with a local file — so making this path frictionless matters more than a headline feature.\n\n### ClickHouse: observability, Postgres, and espresso\nClickHouse published a dense week of content. The [clickhousectl v0.4.0](/release/rel_TjNSDDgxv2yyWHoMT5gyZ-what-s-new-in-clickhousectl-v0-4-0) release adds horizontal autoscaling for Cloud services, ClickPipe schema discovery, and opt-out anonymous telemetry — practical ops improvements for teams running ClickHouse at scale. Two blog posts tackle real-world comparisons: [Benchmarking NVMe-backed Managed Postgres](/release/rel_49a8S0V_pPaCRhscGv4zn-benchmarking-nvme-backed-managed-postgres-planetscale-and-clickhouse) against PlanetScale on identical hardware shows ClickHouse delivering up to 54% higher throughput with lower latency, while [Choosing Between ClickStack and Grafana](/release/rel_bMWCGxkVRDPwFyeE8O2wb-choosing-between-clickstack-and-grafana-for-clickhouse-observability) for ClickHouse observability walks through the trade-offs. On the lighter side, [instrumenting an espresso machine with OpenTelemetry](/release/rel_8T1uJTyAwLVR1iIgbTVMg-instrumenting-my-espresso-machine-with-opentelemetry) is a delightful case study: an ESP32 streams shots and house sensors into ClickHouse Cloud, with ClickStack and an LLM agent turning every pull into a queryable event. Finally, a customer story from [Fetch](/release/rel_ddO3Lrequx5CK7c3K9YO2-how-fetch-built-its-flexible-agentic-analytics-tool-on-clickhouse-cloud) shows how they built FAST, a conversational BI agent on ClickHouse Cloud that lets non-technical teams query 150 billion rows a day in plain language. The [strict memory overcommit](/release/rel_7vFRTHrthT2nw_EuM9Dy9-why-strict-memory-overcommit-matters-for-postgres) post for Managed Postgres is a crisp technical explainer: turning an instance-wide OOM restart into a single failed query, with no measurable throughput cost.","releaseIds":["rel_ehffs5qZ20ftAbF9XIQ_j","rel_13Na5IoyOdsWqwUeHwhLM","rel_TjNSDDgxv2yyWHoMT5gyZ","rel_49a8S0V_pPaCRhscGv4zn","rel_bMWCGxkVRDPwFyeE8O2wb","rel_8T1uJTyAwLVR1iIgbTVMg","rel_ddO3Lrequx5CK7c3K9YO2","rel_7vFRTHrthT2nw_EuM9Dy9"],"releaseCount":8,"generatedAt":"2026-08-03T06:18:55.616Z","releases":[{"id":"rel_ehffs5qZ20ftAbF9XIQ_j","title":"1.13.16 (core) / 0.29.16 (libraries)","path":"/release/rel_ehffs5qZ20ftAbF9XIQ_j-1-13-16-core-0-29-16-libraries","org":{"slug":"dagster","name":"Dagster"},"importance":null},{"id":"rel_13Na5IoyOdsWqwUeHwhLM","title":"v1 local file ingestion is now the default","path":"/release/rel_13Na5IoyOdsWqwUeHwhLM-v1-local-file-ingestion-is-now-the-default","org":{"slug":"tinybird","name":"Tinybird"},"importance":null},{"id":"rel_TjNSDDgxv2yyWHoMT5gyZ","title":"What's new in clickhousectl v0.4.0","path":"/release/rel_TjNSDDgxv2yyWHoMT5gyZ-what-s-new-in-clickhousectl-v0-4-0","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_49a8S0V_pPaCRhscGv4zn","title":"Benchmarking NVMe-backed Managed Postgres: PlanetScale and ClickHouse","path":"/release/rel_49a8S0V_pPaCRhscGv4zn-benchmarking-nvme-backed-managed-postgres-planetscale-and-clickhouse","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_bMWCGxkVRDPwFyeE8O2wb","title":"Choosing Between ClickStack and Grafana for ClickHouse Observability","path":"/release/rel_bMWCGxkVRDPwFyeE8O2wb-choosing-between-clickstack-and-grafana-for-clickhouse-observability","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_8T1uJTyAwLVR1iIgbTVMg","title":"Instrumenting my espresso machine with OpenTelemetry","path":"/release/rel_8T1uJTyAwLVR1iIgbTVMg-instrumenting-my-espresso-machine-with-opentelemetry","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_ddO3Lrequx5CK7c3K9YO2","title":"How Fetch built its flexible agentic analytics tool on ClickHouse Cloud","path":"/release/rel_ddO3Lrequx5CK7c3K9YO2-how-fetch-built-its-flexible-agentic-analytics-tool-on-clickhouse-cloud","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null},{"id":"rel_7vFRTHrthT2nw_EuM9Dy9","title":"Why strict memory overcommit matters for Postgres","path":"/release/rel_7vFRTHrthT2nw_EuM9Dy9-why-strict-memory-overcommit-matters-for-postgres","org":{"slug":"clickhouse","name":"ClickHouse"},"importance":null}]}}