[client] Handle test server empty history when waiting for workflow result
(#902)
[common] Export and deprecate error helpers (#901)
Fixes a breaking change accidentally introduces in 1.4.0 where some rarely used utility functions were deleted.
Improve regex for extracting source map (#899)
Addresses reported issue by userr where regex caused RangeError: Maximum call stack size exceeded when parsing their
workflow bundle.
client] Fix default gRPC keepalive settings not applied
(#906)nyc-test-coverage] Delay checking for coverage until helpers run because coverage is undefined when Jest
starts (#910)worker] Avoid a race in shutdown hooks (#918)core] Ignore cancels of StartFailed state in child workflows, see: https://github.com/temporalio/sdk-core/pull/413Don't request eager activities when worker started with no remote activities
Actual fix made in this Core SDK PR: https://github.com/temporalio/sdk-core/pull/429
[client] The experimental WorkflowHandle.fetchHistory function can now be used to easily obtain a single
Workflow execution's history (#974).
[client] Introduced (experimental) high level API to list workflows (#942,
#974):
for await (const workflowInfo of client.workflow.list({ query: 'WorkflowType="MySuperCoolWorkflow"' })) {
console.log(`${workflowInfo.workflowId} ${workflowInfo.runId}`);
}The same API can also be used to efficiently obtain a list of workflows histories. Multiple histories are fetched from
the server in parallel (up to concurrency, defaults to 5), which may improve performances.
for await (const { workflowId, history } of client.workflow.list().intoHistories({ concurrency: 10 })) {
// ...
}[client] Added (experimental) high level API to work with Schedules (#937,
#960):
// Define a schedule that will start workflow 'RefreshClientTableWorkflow` every day at 5 AM and 1 PM
await client.schedule.create({
scheduleId: `refresh-client-table-every-morning`,
spec: {
calendars: [{ hour: [5, 13] }],
},
action: {
type: 'startWorkflow',
workflowType: 'RefreshClientTableWorkflow',
taskQueue,
},
});Note that Schedules requires Temporal version 1.18 or later.
[core] Core's (experimental) telemetry options are now more configurable (#963,
#977). Notably, filters can now be specified independently
for logging (applicable to both console and forward loggers) and tracing. Function makeTelemetryFilterString
can be used to easily build filter strings. Also, OTel metrics export interval can now be modified (defaults to 1
second).
Note: the TelemetryOptions interface has changed quite a bit. Using appropriate new options is highly recommended.
Backward compatibility for legacy options is provided, to the extent possible, but these legacy options have been
deprecated.
[client] WorkflowClient now supports a simpler way to define interceptors (#956).
Interceptors should now be provided as an array of interceptor object, rather than an array of factory to those
objects under a field named calls. Former definition syntax is still supported, though deprecated.
BEFORE
interceptors: {
calls: [
(workflowId) => {
create(...) => { ... }
}
]
}AFTER
interceptors: [
{
create(...) => { ... }
}
][worker] Introduced an experimental API to efficiently replay a large number of workflow histories. Teams may
notably use this API to validate that changes to their workflow code will not cause non-determinism errors on existing
workflow instances, before rolling out these changes to production (#920,
#974).
EXAMPLE USAGE
const histories = client.workflow.list({ query: 'WorkflowType="MyWorkflow"' }).intoHistories({ concurrency: 10 });
const replayResults = await Worker.runReplayHistories(
{
workflowsPath: require.resolve('./workflows'),
// ...
},
histories
);
console.log(`Found ${replayResults.errors.length} replay errors`);Added activity_task_received metric (#439)
[workflow] Don't fail workflow task if a query handler was not found (#932).
[worker] Wait for worker shutdown if runUntil promise throws (#943).
Previously, Worker.runUntil would not wait for worker to complete its shutdown if the inner fnOrPromise threw an
error. Now, it will always wait for both worker shutdown AND the inner fnOrPromise to resolve. If either one throw
an error, then that error is rethrown. If both throw an error, a CombinedWorkerRunError will be thrown instead,
with a cause attribute containing both errors.
The (experimental) FailureConverter type now receives its PayloadConverter through an argument on convertion
methods, rather than through an option supplied at construction time (#936).
This provides a more predictable behaviour in the common case of using the default failure converter. More over,
FailureConverter.errorToFailure function's return type has been lossen, so that it supports greater customization on
user side (#927)
[client] ConnectionOptions.connectTimeout is now being applied correctly (#954).
[workflow] Properly encode memos in makeContinueAsNewFunc (#955).
They were previously not encoded at all, resulting in a failure due to invalid data.
[worker] Activity metric scheduled_to_start_latency now reports the time from the schedule time of the
current attempt to the start time of that same attempt, instead of the time elapsed since the initial schedule time
(#975). This new definition aligns with other SDKs and is
more useful from a monitoring perspective.
[workflow] Previously, condition(fn, 0) was incorrectly handled the same as condition(fn), meaning that the
function would block indefinitely and would return nothing once fn evaluated to true. It now behaves the same as
condition(fn, 1), ie. the function will sleep for a very short time, then return true if fn evaluates to true,
or false if timeout reaches its expiration (#985).
[core] Fixed some non-deterministic behaviour in workflows containing local activities, due to heartbeats
being incorrectly counted as logical workflow tasks (#987).
[core] core-bridge has been refactored so that it does not retain static references to custom TypeScript error
constructors (#983). This change is part of an ongoing effort
to resolve multiple issues observed by some users in execution of their unit tests based on sdk-typescript, notably in
conjunction with Jest, Mocha and Vitest.
[worker] The default log function now write errors using process.stderr.write rather than console.error
(#940). This avoids complains by some test runners.
[debugger] Log errors comming from VS Code debugger (#968)
Bug Fixes in Core SDK:
core] Use correct property name for OTel metrics export interval in core-bridge (#993).[workflow] (Experimental) Introduced a major optimization to the workflow runtime (#951).
This optimization allows the worker to reuse execution context across workflows, without compromising the safety of the deterministic sandbox. Some initial performance tests have demonstrated reduction of RAM usage by as much as 66%, and reduction of CPU usage by up to 50%.
To enable this feature, add reuseV8Context: true to your WorkerOptions.
[workflow] Added workflowInfo().startTime and workflowInfo().runStartTime. (#1031)
[workflow] Added support for default workflow handlers (#1038).
A workflow bundle may opt-in to receive requests for non-registered workflow types by exporting a default function:
export default async function (...args: unknown[]): Promise<unknown> {
const { workflowType } = workflowInfo();
// ...
}[workflow] Added support for default signal handlers (#1038).
A workflow function may opt-in to receive requests for non-registered signals with:
setDefaultSignalHandler((signalName: string, ...args: unknown[]) => {
// ...
});[worker] It is now possible to launch workers in debug mode by setting environment variable TEMPORAL_DEBUG=true (#1031).
A recent release of @grpc/grpc-js has been causing multiple issues:
We pinned our dependencies on @grpc/grpc-js to 1.7.3 (#1025). This pin will be removed once we confirm that the upstream project is stable again for our usage.
[client] Multiple small changes to the experimental Schedules API. (#1028, #1032, #1009)
[workflow] instanceof on WorkflowInfo fields now works as expected (#1031, #659)
[create-project] create-project now works correctly on Node 18 (#995)
[core] Fixed incorrect calculation of schedule-to-start timeouts on local activities (#450).
[core] Fixed some rare case where sdk-core would panic on unexpected history fetching responses from the server (#468).
[core] Fixed some rare case where an activation completion might get blocked if fetching history pages failed (#478).
:boom: [worker] The experimental Worker.runReplayHistories method, which allows efficient replay of a large number
of workflow histories, now returns an AsyncIterableIterator (#1067)
EXAMPLE USAGE
const histories = client.workflow.list({ query: 'WorkflowType="MyWorkflow"' }).intoHistories({ concurrency: 10 });
const replayResults = Worker.runReplayHistories(
{
workflowsPath: require.resolve('./workflows'),
// ...
},
histories
);
for await (const result of replayResults) {
const { workflowId, runId, error } = result;
// error is either undefined, a ReplayError, or a DeterminismViolationError
}:boom: [worker] WorkerOptions.shutdownGraceTime no longer forcefully shuts the worker down. Now, when shutdownGraceTime passes, the worker just sends Cancellation to running Activities.
Set WorkerOptions.shutdownForceTime to force shutdown. (#1072)
:boom: [testing] Use Temporal CLI to power local test environment (#1077)
Fail Workflow on WorkflowExecutionAlreadyStartedError (#1068)
[create-project] While fixing dependencies on a newly instantiated project, we now recursively search for
package.json and tsconfig.json (#1089 thanks to @jhubbardsf 🙏)
[create-project] Remove the .post-create file (if it exists), before committing to git (#1018)
:boom: Completetly removed support for Node versions <= 14.17. Lot of our dependencies were already requiring Node 14.18+ anyway. (#1070)
Load package abort-controller as a polyfill rather than a complete substitution.
This will ensure using native implementation of that class in Node 16+ (#1070)
:boom: This change might cause TypeScript to warn about incompatible types when working with libraries that are using
custom type definitions for the AbortSignal interface.
import type { AbortSignal as FetchAbortSignal } from 'node-fetch/externals';
// ...
const response = await fetch(url, { signal: Context.current().cancellationSignal as FetchAbortSignal });[client] Get rid of experimental AsyncLocalStorage.enterWith call (#1080)
[core] Fix slot metrics appearing to be off-by-one because of reservation (#479)
[core] Fix misnamed metric workflow_task_execution_failed (#481)
[core] Added an internal patching mechanism for adjusting SDK behavior based on what version of the SDK has previously processed the workflow. Note that this new mechanism requires server Temporal 1.20.0 or above. Any change that depends on such an internal patch will not be effective with older servers. (#482)
:boom: [core] Make activity (and child workflow) type / id mismatches will results in a non-deterministic change. That means that renaming an activity or a child workflow will now require a patch. Note that this change has been gated with an internal patch to avoid suddently failing with nondeterminism errors on older activity rename (see details above) (#475, #482)
[core] Auto-fail new workflow tasks which encounter a problem upon their application, but before any activation has been issued to lang. This may fix some scenarios where previously a WFT would simply time out. (#482)
@jhubbardsf 🙏)workflow] Fix internal patches NDE on replay (#1106)null payload data in historyFromJSON (#1109)core] Fix issues with pagination and event loss in some edge cases (#537 + #538)core] Fix child wf state machine returning fatal instead of NDE (#536)@maxgurewitz 🙏)[worker] Add support for worker versioning (#1146).
Worker versioning is available from server version 1.21 (if enabled in dynamic configuration).
:warning: Experimental - While the feature is well tested and is considered functionally stable, the SDK APIs are considered experimental.
To use worker versioning with the TypeScript SDK, use the following steps:
import { Client } from '@temporalio/client';
import { Worker } from '@temporalio/worker';
const buildId = 'id-generated-from-continuous-integration';
// Deploy new workers, opt them in to versioning.
const worker = await Worker.create({
workflowsPath: require.resolve('./workflows'),
buildId,
useVersioning: true,
// ...
});
// ...
// In a separate process, when all workers are up, update the build id compatibility for the task queue.
const client = new Client({
/* options */
});
// If the current version is incompatible with the previous ones:
await client.taskQueue.updateBuildIdCompatibility('my-task-queue', {
operation: 'addNewIdInNewDefaultSet',
buildId,
});
// Or, if the current version is compatible with a previous one:
await client.taskQueue.updateBuildIdCompatibility('my-task-queue', {
operation: 'addNewCompatibleVersion',
buildId,
existingCompatibleBuildId: 'some-other-build-id',
});
// Check if workers are reachable before retiring them (even if their build ids are associated with multiple task
// queues):
const { buildIdReachability } = await client.taskQueue.getReachability({ buildIds: ['some-other-build-id'] });
const { taskQueueReachability } = buildIdReachability['some-other-build-id'];
for (const [taskQueue, reachability] of Object.entries(taskQueueReachability)) {
if (reachability.length > 0) {
if (reachability[0] === 'NotFetched') {
// We asked the server for too many reachability entries (build ids or task queues),
// This build id / task queue reachability should be fetched in another request.
// Fetch this information here or later...
} else {
console.log('Build id still reachable on:', taskQueue);
}
}
}
// Check if build id is reachable...[worker] Add support for using multiple concurrent pollers to fetch Workflow Tasks and Activity Tasks from Task Queues (#1132).
The number of pollers for each type can be controlled through the WorkerOptions.maxConcurrentWorkflowTaskPolls and WorkerOptions.maxConcurrentActivityTaskPolls properties. Properly adjusting these values should allow better filling of the corresponding execution slots, which may signficiantly improve a Worker's throughput. Defaults are 10 Workflow Task Pollers and 2 Activity Task Pollers; we however strongly recommend tuning these values based on workload specific performance tests.
Default value for maxConcurrentWorkflowTaskExecutions has also been reduced to 40 (it was previously 100), as recent performance tests demonstrate that higher values increase the risk of Workflow Task Timeouts unless other options are also tuned. This was not problem previously because the single poller was unlikely to fill all execution slots, so maximum would rarely be reached.
[workflow] The reuseV8Context worker option is no longer marked as experimental (#1132). This is a major optimization of the Workflow sandboxing runtime; it allows the worker to reuse a single execution context across Workflow instances, without compromising the safety of the deterministic sandbox. It significantly reduces RAM and CPU usage. The formula used to auto-configure maxCachedWorkflows has also been reviewed to reflect a lower memory usage requirement when reuseV8Context is enabled.
At this point, you still need to opt-in to this feature by adding reuseV8Context: true to your WorkerOptions, as we believe most teams should reconsider their workers's performance settings after enabling this option.
:boom: Note that we are planing enabling this option by default starting with 1.9.0. If for some reason, you prefer to delay enabling this optimization, then we recommend that you explicitly add reuseV8Context: false to your worker options.
We now provide out-of-the-box log support from both Workflows and Activity contexts (#1117, #1138)).
For Workflows, the logger funnels messages through the defaultWorkerLogger sink, which itself defaults to forwarding messages to Runtime.instance().logger.
Example usage:
import * as workflow from '@temporalio/workflow';
export async function myWorkflow(): Promise<void> {
workflow.log.info('hello from my workflow', { key: 'value' });
}For Activities, the logger can be accessed as Context.log. It defaults to Runtime.instance().logger, but may be overridden by interceptors (i.e. to set a custom logger). ActivityInboundLogInterceptor is still installed by default, adding enriched metadata from activity context on each log entry.
Example usage:
import * as activity from '@temporalio/activity';
export async function myActivity(): Promise<void> {
const context = activity.Context.current();
context.log.info('hello from my activity', { key: 'value' });
}:boom: Protect against 'ms' durations errors (#1136). There have been several reports of situations where invalid durations resulted in unexpected and hard to diagnose issues (e.g. can you predict what const bool = condition(fn, '1 month') will do?). We now provide type definitions for "ms-formatted strings" through the newly introduced Duration type, which is either a well-formed ms-formatted string or a number of milliseconds. Invalid ms-formatted-strings will also throw at runtime.
Note: this might cause build errors in situations where a non-const string value is passed somewhere we expect a Duration. Consider either validating and converting these strings before passing them as Duration, or simply cast them to Duration and deal with runtime exceptions that might be thrown if an invalid value is provided.
[workflow] Clone sink args at call time on Node 17+ (#1118). A subtle aspect of Workflow Sinks is that calls are actually buffered and get executed only once the current Workflow activation completes. That sometime caused unexpected behavior where an object passed as argument to a sink function is mutated after the invocation.
On Node.js 17+, we now clone sink arguments at call time, using structuredClone. While this adds some runtime overhead, it leads to more predictable experience, as well as better exceptions when passing non-transferrable objects to a sink.
[core] Add the sticky_cache_total_forced_eviction metric (Core #569)
[client] Throw more specific errors from Client APIs (#1147)
core] Metrics that should be produced by the SDK Core's internal Client would previously not get emitted. This has been fixed. (#1119)client] Fix incorrect schedule spec boundaries checks on hour and day of month (#1120)workflow] We now throw more meaningful errors when Workflow-only APIs are used from non-Workflow context, and some other situations. (#1126)instanceof checks from SDK, and replaced them by XxxError.is(...) checks, based on the presence of a symbol property. We believe this should help resolve most of the problems that previously been observed when multiple copies or different versions of SDK packages are installed in a same project ((#1128)).workflow] Make Local Activity timeouts in ActivityInfo match those of non-Local Activities (#1133, Core #569).workflow] Ensure payload converters keep Uint8Array type equality (#1143)core] Don't increment terminal command metrics when replaying (Core #572)core] Fix start-to-close local activity timeouts not being retriable like they should be (#Core 576)Remove instanceof Error checks from is methods (#1162). This fixes various regressions observed when running Workflow tests with Jest.
Add the VersioningIntent parameters on the experimental worker versioning feature (#1156). Also, all values of the ReachabilityType type has been converted to SCREAMING_CASE.
[workflow] Make workflows.log()'s attributes argument optional (#1159).
[workflow] Attach workflows.log()'s attributes to every workflow log message (#1159).
[workflow] Add support for URL/URLSearchParams inside the workflow sandbox (#1173)
[worker] Export the WorkerStatus interface (#1184).
Use custom symbol-based implementation of instanceof for all of our custom error classes (#1166). This is a complete rework of the fixes introduced in 1.8.0 and 1.8.1, both of which turned out to be insufficient. instanceof now works correctly both across execution contexts and when running tests with Jest.
:boom: The static is function introduced previously on some of our error classes is no longer required, as the instanceof operator itself now behave correctly; these is functions have therefore been removed.
[client] Make action.workflowId optional on Schedule update (#1176)
[activity] heartbeatTimeoutMs is now correctly set on Activity's Context.current().info. The currentAttemptScheduledTimestampMs property has also been added to that data structure (#1187)
[workflow] The Workflow Bundler is now smarter regarding requiring files with incorrect or missing file extensions (#1186). Thanks to @GauBen 🙏.
[workflow] Fix incorrect values of workflowInfo.historyLength and workflowInfo.unsafe.isReplaying as reported in out-of-sandbox log messages, and as argument of sink function implementations (#1181).
[workflow] Sink functions configured with callDuringReplay = false are no longer invoked from a replay-only worker (i.e. Worker.runReplayHistories()); it was previously possible for these to get called in some cases on the very last Workflow Task (#1181).
[workflow] Encoding and decoding of protobuf JSON payloads is now working in workflow context. This was previously failing, due to the fact that an underlying library depends on the Buffer class, which is not available inside the workflow sandbox (#1170). Thanks to @antlai-temporal 🙏.
[core] Improve warning message on error response to completion (temporalio/sdk-core#581) Thanks to @dandavison 🙏.
[core] Fix abandoned children nondeterminism if they complete (temporalio/sdk-core#580).