Schema changes
Warning
Before upgrading your Temporal Cluster to v1.31.0, you must upgrade core and visibility schema to the following:
- Core:
- MySQL schema
v1.19
- PostgreSQL schema
v1.19
- SQLite schema
v1.11
- Visibility:
- Elasticsearch schema
v14
- MySQL schema
v1.14
- PostgreSQL schema
v1.14
Please see our upgrade documentation for the necessary steps to upgrade your schemas.
Please see CHASM section below for more information about the core schema change if you have a custom persistence implementation.
Temporal database schema details
MySQL v1.19, PostgreSQL v1.19, SQLite v1.11 — adds a new current_chasm_executions table for CHASM separate archetype ID spaces:
CREATE TABLE current_chasm_executions(
shard_id, namespace_id, business_id, archetype_id,
run_id, create_request_id, state, status, start_version,
start_time, last_write_version, data, data_encoding,
PRIMARY KEY (shard_id, namespace_id, business_id, archetype_id)
);
Visibility schema details
MySQL v1.14, PostgreSQL v1.14 — adds TemporalExternalPayloadSizeBytes and TemporalExternalPayloadCount as generated columns with indexes.
Elasticsearch v14 — new index template version with TemporalExternalPayloadSizeBytes and TemporalExternalPayloadCount fields.
CHASM persistence schema change (custom persistence implementations only)
CHASM framework now supports separate businessID spaces for different archetypes and includes corresponding schema changes for SQL core databases. This is achieved by storing multiple current mutable state records in the database, one for each archetype.
If you maintain a custom persistence implementation (Cassandra-based or SQL-based), you must update it to support separate businessID spaces as well. All current mutable state related persistence requests now include a new ArchetypeID field, and your persistence implementation should read or update the current mutable state record for that archetype. Please refer to https://github.com/temporalio/temporal/pull/8907 (Cassandra) or https://github.com/temporalio/temporal/pull/8915 (SQL) for sample changes.
Principal Attribution
Adds a server-computed, immutable Principal field to workflow history events, providing trustworthy attribution for "who did this?". Unlike the client-supplied identity field, Principal is derived from authenticated context and cannot be spoofed.
Principal has a Type / Name pair (e.g. jwt/alice@company.com, temporal/internal) and is surfaced in workflow history, CLI, Web UI, and history exports.
The default Authorizer populates Principal from the JWT sub claim. Users with custom Authorizer implementations can populate Principal by setting the new Principal field on authorization.Result.
To enable this feature, set the dynamic config system.enablePrincipalAttribution to true.
Cloud IAM auth for SQL datastores
https://github.com/temporalio/temporal/pull/9879 introduced new passwordCommand config option for SQL datastores as an alternative to the static password field. When set, Temporal executes the specified command and uses its stdout as the database password, re-running the command on each new physical connection so short-lived credentials stay fresh.
This enables IAM-based auth for cloud-managed databases (e.g. AWS RDS, GCP Cloud SQL) by invoking the provider's CLI. For example, configuring the passwordCommand to:
aws rds generate-db-auth-token for AWS RDSgcloud sql generate-login-token for GCP Cloud SQL
Supported for MySQL and PostgreSQL. Token lifetime is managed through existing config maxConnLifetime.
Nexus
💥 Overhaul Nexus error model
Complete rework of how Nexus errors are handled and converted (https://github.com/temporalio/temporal/pull/9290).
This change enables Nexus handler errors and operation errors to convey their own messages as opposed to being wrappers for an underlying cause. The change was made to better support Nexus in all languages. There may be slight changes to errors / exception structures in the SDKs.
Remove Nexus feature flag and make Nexus work out of the box
Nexus is now always enabled (https://github.com/temporalio/temporal/pull/9512).
Use token based routing by default with token-based callback routing (https://github.com/temporalio/temporal/pull/9513).
Nexus caller timeouts
Support schedule-to-start and start-to-close timeouts for Nexus operations. SDK upgrade required.
Bug fixes
Metrics
Worker Versioning
Worker Versioning GA
Worker Deployment APIs are now fully GA. The following APIs have been in Public Preview since server v1.28.0. They are now considered GA and users can rely on the signature and behavior consistency going forward.
DescribeWorkerDeploymentDeleteWorkerDeploymentListWorkerDeploymentsSetWorkerDeploymentManagerDescribeWorkerDeploymentVersionDeleteWorkerDeploymentVersionSetWorkerDeploymentCurrentVersionSetWorkerDeploymentRampingVersionUpdateWorkerDeploymentVersionMetadata
⚠️💥 Sunsetting Worker Versioning V1 (Version Sets) and V2 (Assignment Rules)
The following APIs which have been deprecated since v1.28.0 are now officially sunsetted and their support will be removed in the next server version (v1.32.0). Users should migrate their applications to use the Worker Deployment and Worker Deployment Version APIs instead.
APIs to be removed in v1.32.0:
UpdateWorkerBuildIdCompatibilityGetWorkerBuildIdCompatibilityUpdateWorkerVersioningRulesGetWorkerVersioningRulesGetWorkerTaskReachability
New Experimental APIs
The following APIs are added in this release and are in Pre-Release stage as of now. At this stage, the APIs are considered experimental and may see breaking changes in the future:
CreateWorkerDeploymentCreateWorkerDeploymentVersionUpdateWorkerDeploymentVersionComputeConfigValidateWorkerDeploymentVersionComputeConfig
Other Changes
- Add new ContinueAsNew versioning behavior that continues workflows on the ramping version.
- Fixed bug around stuck workflow after reset when WorkflowUpdate was used.
- Add deployment name and build ID as separate labels on backlog metrics.
- Rate-limit worker deployment read APIs and move
ListWorkerDeployments to the visibility quota.
- Cache task queue routing info in the History service to reduce RPC fan-out.
- Version transition from Drained/Inactive to Draining state upon workflow start/moved via Versioning Override is disabled by default until the implementation is optimized in the next release.
- Some other minor bug fixes or improvements.
CHASM
- CHASM framework is enabled by default in this release, but applications built on top of it are NOT enabled by default to allow safe downgrades.
- CHASM framework now supports separate
businessID spaces for different archetypes and includes corresponding schema changes for SQL core databases. See the Schema Changes section above if you maintain a custom persistence implementation.
Standalone Activities
- Standalone Activities (in public preview) allow activities to run independently of workflows. This feature is gated by the
activity.enableStandalone dynamic config flag, which is disabled by default. Set it to true to enable.
- New APIs and capabilities:
DeleteStandaloneActivity API for deleting a standalone activity execution.
- Durability improvements: server-generated request IDs are now preserved across restarts, a standby task discard handler has been added, and the 1-day retention limit has been removed.
PollActivityTaskQueueResponse now includes additional fields needed by workers running without a parent workflow, such as currentAttemptScheduledTime and namespace. Termination and cancellation identity is now propagated through failure info.- Multiple timing and retry bugs have been fixed, including the Start-To-Close timeout bug and its metric timestamps, incorrect
RetryState for schedule timeouts, and TimerTaskStatus reset issues. Heart beating, RecordActivityTaskStarted, and various error codes also received fixes.
- Internal task queue name validation has been added for standalone activities and all user-facing APIs.
Task Queue Priority and Fairness GA
- Task queue priority and fairness features are now GA.
- As part of that, the new matcher component is enabled by default, which means priority keys are respected by default. The new matcher is fully compatible and switching should be transparent, but if you have any issues you can switch back by setting
matching.newUseMatcher to false (affected task queues will reload immediately).
- As before, fairness can be enabled on a task queue, namespace, or cluster level with the
matching.enableFairness dynamic config. Migration between fairness-enabled and fairness-disabled queues is enabled by default.
Cassandra 5 support
- Temporal Server now supports Cassandra
5.0.4 and later.
- Cassandra 5 versions earlier than
5.0.4 are not supported.
Helpful links to get you started with Temporal
Temporal Docs
Server
Samples Server
Helm Chart
Docker images
Server
Admin-Tools
Full Changelog: https://github.com/temporalio/temporal/compare/v1.30.4...v1.31.0
