npx @buildinternet/releases get livekit-agents-jsKeep healthy STT providers available when fallback streams close with transcripts in flight. - #2381 (@swayamg20)
Fail fast with a clear error when a DuplexModel such as GPT-Live is started under a text simulation, which has no audio, instead of timing out on the first reply. - #2481 (@u9g)
Reject late inline AgentTasks on an outgoing activity so handoff and shutdown do not deadlock while draining non-cancellable tools. - #2488 (@swayamg20)
Place recorded audio where the sink reports it actually played, instead of inferring it from the moment playback finished. - #2327 (@longcw)
Stop session.interrupt({ force: true }) from crashing the process with SpeechHandleCircularWaitError when called inside a function tool. - #2444 (@chenghao-mou)
Wait for agent task handoffs during session shutdown, including concurrent agent updates, without leaking activities or waiting on the task's own tool. Preserve transfer errors, suppress tool replies during shutdown, and make warm-transfer room-disconnect failures explicit. - #2472 (@chenghao-mou)
Settle pending AMD detection when session shutdown starts, release waiting callbacks, and preserve results if interruption or reply authorization fails. - #2451 (@chenghao-mou)
Add DuplexModel and DuplexSession for full-duplex speech providers. Agent and AgentSession accept duplex models through an adapter that segments continuous audio and aligns transcripts. Preserve overlapping speech and collect usage reported during realtime session shutdown. - #2471 (@chenghao-mou)
Preserve trailing inference STT transcripts until transcript inactivity, session closure, or socket closure after input ends. - #2465 (@chenghao-mou)
Resolve transcript-only AMD greetings at the endpointing backstop, preserve timeout transcripts, and allow null to force session LLM or STT reuse. - #2400 (@rosetta-livekit-bot)
Safely describe option objects in uploaded session reports. - #2431 (@rosetta-livekit-bot)
Handle rejected promises in agent shutdown and room state updates. Close Silero VAD output and release its resampler when inference fails. - #2446 (@chenghao-mou)
Stop retaining consumed audio frames in the forwarding result. - #2433 (@rosetta-livekit-bot)
Serialize AgentConfigUpdate.instructions as the rendered string in toJSON, matching the Python SDK and the proto encoder. - #2467 (@chenghao-mou)
Add the GPT-Live full-duplex voice plugin with Responses and client delegation. - #2474 (@chenghao-mou)
Respect empty synchronized transcripts when committing interrupted replies to chat history. - #2464 (@chenghao-mou)
Clarify the slow-inference warning message to name VAD as the source. - #2453 (@claude)
Emit the full OpenTelemetry GenAI semantic conventions on agent spans. - #2411 (@theomonnom)
Spans now carry the standard gen_ai.* attributes — operation, provider, request/response
model, per-modality token usage, cached tokens, finish reasons, time-to-first-chunk, tool
name/type/call id, and the gen_ai.input.messages / gen_ai.output.messages /
gen_ai.system_instructions / gen_ai.tool.definitions content payloads — so Datadog Agent
Observability, Langfuse and any other backend that reads the conventions (OTel v1.37+)
renders a LiveKit trace without a custom mapping. The session maps to invoke_workflow, an
agent turn to invoke_agent, inference to chat (realtime turns to generate_content on
their own nested span), and tool execution to execute_tool.
gen_ai.provider.name now reports the registry spelling the convention requires. Plugins
expose provider either as a display name (MistralAI, AWS Bedrock, Vertex AI) or as the
client's base-URL host (api.openai.com, api.anthropic.com); both are normalized, so
openai, anthropic, mistral_ai, aws.bedrock, gcp.vertex_ai, gcp.gemini, x_ai,
groq and perplexity are recognized by GenAI backends. A provider outside the registry
keeps its own id, which the convention allows.
Conversational content reaches every configured exporter, as before. To withhold it from a
third-party pipeline while LiveKit Cloud keeps receiving it, pass allowPii: false:
telemetry.setTracerProvider(provider, { registerSpanProcessor, allowPii: false });or set LIVEKIT_TELEMETRY_ALLOW_PII=0 when the framework adopts the ambient OpenTelemetry
provider and there is no call site. What LiveKit Cloud receives stays governed by the
project's PII setting in the dashboard; when that mandates redaction, PII is withheld from
every destination and allowPii does not weaken it. Content can be dropped entirely with
telemetry.genAI.setCaptureContent(false) or
OTEL_INSTRUMENTATION_GENAI_CAPTURE_MESSAGE_CONTENT=false.
Handle microphone track replacement for an existing RoomIO publication. - #2363 (@rosetta-livekit-bot)
Retry connection pool prewarming after a previous attempt fails or completes. - #2378 (@rosetta-livekit-bot)
fix(voice): cancel a parked preemptive generation when pausing speech scheduling so agent handoffs cannot deadlock - #2398 (@rosetta-livekit-bot)
fix(voice): cancel a speech that scheduleSpeech refuses while scheduling is paused, so its speech task does not stay parked and block the next drain()/pause() and AgentSession.close() - #2372 (@chenghao-mou)
Add an onSessionEnd lifecycle callback and configurable timeout before internal session report cleanup. Flush final OTEL logs after shutdown callbacks with a bounded timeout. Apply Python's connection, total, and retry policy to session report uploads. - #2387 (@chenghao-mou)
Wait for session report cleanup before applying the process shutdown timeout, force-stop jobs that exceed memory limits, preserve logs queued during a final flush, and keep cleanup details and session metadata safe for logging.
Honour LIVEKIT_OBSERVABILITY_URL when resolving the observability endpoint, keeping the override's scheme, port, and base path so a plaintext or non-default-port collector is reachable.
The telemetry exporters now take observabilityUrl (a full base URL) in place of cloudHostname (a bare host that always implied https). cloudHostname is deprecated but still accepted on setupCloudTracer, uploadSessionReport, SimpleOTLPHttpLogExporterConfig, and PinoCloudExporterConfig, and resolves to the same endpoint as before.
Preserve wired VAD speech-end anchors when reporting transcription delay, expose absolute STT speech-end timing, and restore default-VAD speaking state for false-interruption recovery. - #2382 (@rosetta-livekit-bot)
Preserve non-ASCII characters in pretty logs on Windows terminals. - #2367 (@chenghao-mou)
Ignore late adaptive interruption responses after the agent speech that requested them ends. - #2417 (@chenghao-mou)
Text-mode simulation jobs now send X-LiveKit-Inference-Priority: low on every inference request, overriding the model's configured inferenceClass, so simulation runs are paced into spare quota instead of competing with live voice traffic. Matches the Python SDK. - #2423 (@u9g)
Redact API keys from rejected WebSocket handshake errors. - #2379 (@rosetta-livekit-bot)
Fix WarmTransferTask hanging after a failed participant move when the human agent disconnects. - #2361 (@chenghao-mou)
Remove retired ElevenLabs models from LiveKit Inference STT and TTS. - #2415 (@chenghao-mou)
Close LiveKit Inference STT sessions and sockets when input ends, include the model in WebSocket dials, and stop retrying gateway inactivity closures. - #2413 (@chenghao-mou)
Surface inference gateway status errors from WebSocket handshakes and active STT sessions. - #2412 (@chenghao-mou)
Exclude active agent speech from pauses learned by dynamic endpointing. - #2422 (@rosetta-livekit-bot)
Support Mistral Conversations requests when the chat context ends with an assistant message. - #2376 (@rosetta-livekit-bot)
Timestamp say() transcripts when speech starts so interrupted turns remain chronologically ordered. - #2421 (@rosetta-livekit-bot)
RoomIO now preserves a pre-set session.input.audio and session.output.transcription instead of silently replacing them, matching the existing behavior for session.output.audio and the python implementation. - #2296 (@1egoman)
Preserve adaptive interruption boundaries when agent playout pauses or enters a tool-call thinking gap. - #2290 (@rosetta-livekit-bot)
Add Speechmatics Linden inference STT model support. - #2353 (@rosetta-livekit-bot)
Add Grok 4.3 and Grok 4.5 to the supported xAI inference models. - #2338 (@rosetta-livekit-bot)
Add callerHangupSpeech for deterministic text or custom speech handles during warm transfers. - #2299 (@chenghao-mou)
Export SpeechHandle and deprecate callerHangupInstruction.
fix(voice): avoid AgentTask handoff deadlock during session close - #1776 (@rosetta-livekit-bot)
Report recording chat-item timestamps in Unix seconds and warn that audio redaction may be inaccurate for realtime sessions. - #2348 (@chenghao-mou)
Add Gemini 3.5 Transcribe Live to LiveKit inference and the Google beta plugin. - #2352 (@rosetta-livekit-bot)
stt.SpeechStream now resets its retry budget after every final transcript, so maxRetry
bounds consecutive failures rather than the lifetime of the stream. Providers that recycle
their socket on a fixed interval (such as Gemini Live's 10-minute session cap) previously
exhausted the budget and stopped recognizing on long sessions.
Add the deployed LiveKit Cloud region to every log record. - #2344 (@rosetta-livekit-bot)
Add greetingSpeech for optional answer-time speech during warm transfers. - #2300 (@chenghao-mou)
Include the JavaScript SDK version in Cloud observability session reports. - #2340 (@chenghao-mou)
Send deployment environment in worker registration. - #1677 (@rosetta-livekit-bot)
Prevent delayed cleanup from an interrupted reply from overwriting the agent state owned by a newer reply. - #2341 (@chenghao-mou)
Interrupt superseded queued replies when a new user turn arrives during interrupted speech cleanup. - #2342 (@chenghao-mou)
Serialize inline AgentTasks launched by parallel tool calls. - #2313 (@rosetta-livekit-bot)
Stamp simulation run and job IDs on session telemetry and expose them on SimulationContext. - #2351 (@rosetta-livekit-bot)
Add an optional abortSignal to WarmTransferTask. Aborting stops waiting for a pending dial or ends an active consultation, and run() rejects with the signal reason. - #2295 (@chenghao-mou)
Report the worker protocol version on the /worker endpoint so LiveKit Cloud recognizes agent deployment support. - #1677 (@rosetta-livekit-bot)
lk.pii.* and redact trace exception details when project or session redaction is enabled. Redacted audio uploads now require transcript uploads, Deepgram parse failures retain readable details without logging connection query parameters, and MiniMax task failure payloads no longer appear in exception messages. Dashboards and queries using the previous sensitive trace keys must migrate to their lk.pii.* replacements. - #2100 (@chenghao-mou)⚠️ Remove Rime Arcana model support before Rime's cloud cutoff on August 15, 2026 at 12:00 UTC. Set modelId to coda when you upgrade. The default paths keep their established speakers for voice continuity. - #2253 (@naszzz)
Send every field the agent_session wire format declares for chat items, and emit the telemetry chat item log with the same field names and shapes as livekit/agents. - #2247 (@u9g)
Wait for pending turn finalization when closing a voice session. - #2307 (@rosetta-livekit-bot)
Require local inference 0.2.7. - #2303 (@chenghao-mou)
Skip LiveKit Cloud trace, OTLP and Pino log, session report, and recording uploads after Cloud reports project data recording is disabled. - #2107 (@rosetta-livekit-bot)
Start answering machine detection timeouts after SIP calls are answered and settle when the participant disappears before audio arrives. - #2226 (@rosetta-livekit-bot)
Exclude discarded room audio from reported playback duration. The room audio queue now defaults - #2256 (@rosetta-livekit-bot)
to 200ms, matching Python. Set RoomOutputOptions.queueSizeMs to retain a larger prebuffer for
bursty TTS output.
Add Agent.updateOptions() for swapping STT, VAD, LLM, and TTS models at runtime, including - #2297 (@rosetta-livekit-bot)
explicit null values to disable session fallback, and allow expressive TTS settings to be
updated dynamically on agents and sessions. Only the fields you pass are changed, matching
Agent.update_options() in the Python framework.
chore(deps): update @livekit/rtc-node to 0.13.34 - #2304 (@1egoman)
Ports data streams over to the rust livekit-ffi implementation, replacing the TypeScript one (livekit/node-sdks#697). Data streams v2 brings single-packet streams and DEFLATE compression, roughly doubling throughput.
Behavior changes worth noting for anything reading a stream:
readAll() needs to handle rejections. Both of our stream handlers
(RoomIO.onUserTextInput and RoomSessionTransport.onByteStream) already do.dataStream.maxPayloadByteLength on room.connect.sendText and sendFile accept a new compress option (default true).Preserve spaces in Cartesia inference TTS aligned transcripts. - #2285 (@chenghao-mou)
Commit completed tool outputs before starting the post-tool reply so overlapping turns cannot reuse stale preemptive generations. Preserve tool completion timestamps, pair tool-error outputs with their calls, and normalize unparseable call arguments before saving them. - #2261 (@chenghao-mou)
Add an optional transcriptionTimeout session option and a user_transcription_timeout event, emitted for VAD speech that produces no final transcript. - #2208 (@rosetta-livekit-bot)
Include resolved recording options under session report options instead of as a top-level field. - #2274 (@rosetta-livekit-bot)
Use the active agent endpointing delay while holding pending replies during user speech. - #2276 (@chenghao-mou)
Default null token counts in OpenAI-compatible streaming usage payloads to zero. - #2250 (@rosetta-livekit-bot)
Add ChatContext.remove() for removing chat items by item or ID. - #1963 (@rosetta-livekit-bot)
fix: wait 100ms before the first retry, not 0.1ms - #2219 (@u9g)
intervalForRetry returned 0.1 for the first retry — Python's 0.1 seconds, carried over
without converting to the milliseconds every caller passes to setTimeout/delay. Its other
branch returns retryIntervalMs, so the two return paths were in different units. Measured,
the first retry waited ~3ms (scheduling overhead alone) against Python's 100ms.
This affects every retrying component — LLM, STT, TTS, avatars, and the turn-detector transport — where a first retry reattempted essentially instantly.
Expose prompt-cache creation token counts in LLM metrics and model usage. - #2257 (@rosetta-livekit-bot)
fix(inference): keep a stream retryable until it emits generation - #2219 (@u9g)
retryable was cleared by any chunk reaching the caller, including ones that carry no
output: a usage block, or provider metadata such as the LiveKit inference gateway's
deployment/tier stamp or a Gemini thought signature. The gateway stamps its leading
(contentless) delta, so every streamed response went unretryable from its first chunk — a
mid-stream stall then failed the turn outright rather than retrying, with nothing generated
and nothing for a retry to duplicate. Only failures landing before the very first chunk
still recovered.
retryable is now cleared on text or a tool call, the output a retry would actually repeat.
fix(llm): measure ttft against generation, not the first chunk to arrive - #2219 (@u9g)
The time-to-first-token clock started on whatever chunk arrived first. That was harmless while a stream went unretryable at its first chunk, because a retry then implied nothing had arrived. Now that a contentless chunk keeps a stream retryable, the metadata chunk of a failed attempt starts the clock, and the turn reports a near-zero ttft for a wait that spanned a stall and a retry — so retried turns read as faster than a normal one rather than slower.
The clock now starts on generation, in both the metrics monitor and the voice pipeline's
span. A response that generates nothing continues to report ttftMs as -1.
Support low-priority LiveKit inference requests. - #2268 (@rosetta-livekit-bot)
Add expressive mode: AgentSession({ expressive: true }) injects the TTS provider's markup guide into the LLM prompt so the model emits inline <expr/> delivery markers (emotion, pacing, non-verbal sounds). The markers are lowered to each provider's native syntax before synthesis (Cartesia, Inworld TTS 2, xAI, Fish Audio) and stripped from transcripts, with the segment's leading expression surfaced as the lk.expression transcription attribute. Steer delivery with ExpressiveOptions.speechSteering or override the injected prompt entirely. - #2267 (@tinalenguyen)
Fix worker cleanup after LiveKit connection retries are exhausted. - #1889 (@rosetta-livekit-bot)
Avoid throwing when interrupting protected speech that was already interrupted or completed. - #2252 (@rosetta-livekit-bot)
Hold pending agent replies when user speech overlaps their playout launch. - #2263 (@rosetta-livekit-bot)
fix(agents): stop a terminally failed stream leaving an unhandled rejection - #2219 (@u9g)
LLMStream and STTStream dropped the promise from their fire-and-forget main task, so a
stream that exhausted its retries rejected with no handler attached — reaching the job
process's unhandledRejection hook as a spurious crash report for a failure already
delivered through the error event, and failing any test run that exercises the path.
The task is now awaited inside a try/finally that closes the queue either way, matching
what TTSStream already does.
Avoid dropping realtime turns or resuming agent speech before a paused turn decision settles. - #2204 (@rosetta-livekit-bot)
A turn decision that is cancelled no longer resumes the paused speech, and one that fails no longer leaves the agent's audio output paused indefinitely.
Keep defaulted const fields non-nullable in strict tool schemas. - #2262 (@rosetta-livekit-bot)
Stop interrupting queued speech at the first live speech that disallows interruptions, preserving - #2251 (@rosetta-livekit-bot) it and the queued speech behind it.
Strip markdown emphasis from CJK, kana, Thai, and Korean text before TTS. - #2175 (@rosetta-livekit-bot)
fix: omit interruption telemetry from normal user turns - #2265 (@chenghao-mou)
Forward agent false interruption events through remote sessions. - #2214 (@rosetta-livekit-bot)
Apply tool defaults when models return null sentinels for defaulted schema fields. - #2150 (@rosetta-livekit-bot)
Keep audio input mute independent of overridable attach hooks - #2232 (@toubatbrian)
AgentInput now flips mute via AudioInput.setAttached() before calling onAttached/onDetached, and ParticipantAudioInputStream gates on that state. Subclasses that override lifecycle hooks without super no longer silently break hold mute.
Prewarm LLM provider connections before the first inference request. - #2106 (@rosetta-livekit-bot)
Stop forwarding participant audio while agent input is disabled. - #2222 (@dtran26)
Disable the default AEC warmup for outbound SIP calls while preserving explicit settings. - #2202 (@rosetta-livekit-bot)
Commit realtime tool call outputs to the agent chat context. Previously the realtime path only sent them to the provider and to session.history, leaving agent.chatCtx with function calls that had no matching outputs — which broke action-aware history summarization and agent handoff merges. - #2230 (@tinalenguyen)
Reuse preemptive generations when final transcripts differ only in formatting. - #2209 (@rosetta-livekit-bot)
Add cloud agent id and deployment to telemetry resource attributes (lk.cloud_agent_id / lk.deployment_id) when running on LiveKit Cloud. - #2137 (@jmcclanahan)
Let the inference gateway mint avatar worker tokens instead of signing them locally. - #2152 (@rosetta-livekit-bot)
fix(voice): defer interim transcript interruptions to local VAD when configured - #2159 (@rosetta-livekit-bot)
Deprecate direct dev mode in favor of lk agent dev. - #2194 (@rosetta-livekit-bot)
fix(voice): stop an interrupted reply from muting the session forever - #2127 (@toubatbrian)
A reply interrupted before its audio started playing could leave its pipeline reply task
parked in the post-interrupt waitForPlayout(), which races only the reply's own abort
signal — a signal nothing on the ordinary interrupt path ever fires. The speech scheduling
loop waits on that reply's generation, so _currentSpeech stayed pinned on the interrupted
handle and every later turn was queued but never authorized: the agent went silent for the
rest of the session. On the evidence so far this needs an audio sink whose playback-finished
event the pipeline does not produce itself — remote avatar outputs (DataStreamAudioOutput
and the avatar plugins built on it) and user-supplied AudioOutputs; a plain room output
settled both of the affected waits on its own across six live runs.
SpeechHandle now arms a 5s watchdog when a speech is interrupted (a port of python's
INTERRUPTION_TIMEOUT): if the speech has not finished by then, its tasks are cancelled —
firing exactly the abort signal those waits are already watching — and the handle is marked
done, releasing the scheduler.
Fixes #2065.
Send turn finalization messages when flushing streaming STT with no buffered audio. - #2172 (@rosetta-livekit-bot)
Preserve concrete STT language hints when later transcripts report non-specific language codes. - #2149 (@rosetta-livekit-bot)
fix realtime user turns appearing after the reply they prompted, by placing the user message at the time its turn began rather than at the time the provider delivered the transcript - #2165 (@toubatbrian)
Report speech onset immediately when an inference STT provider detects it server-side. - #2182 (@rosetta-livekit-bot)
stamp a tool call's createdAt when its execution begins rather than when it was parsed off the model stream, so it no longer sorts ahead of the assistant message that requested it - #2171 (@u9g)
Avoid emitting playbackStarted again when room audio resumes mid-segment. - #2184 (@rosetta-livekit-bot)
Report word-aligned transcripts only when the inference STT model and every fallback are verified - #2181 (@rosetta-livekit-bot) to provide word timings.
Handle list-shaped content parts in streamed LLM deltas when stripping thinking tokens. - #2188 (@rosetta-livekit-bot)
Type Cartesia turn-detection parameters in inference STT model options. - #2173 (@rosetta-livekit-bot)
chore(deps): update @livekit/rtc-node to 0.13.33 - #2191 (@theomonnom)
Picks up livekit-ffi 0.12.73 and a fix for the dual package hazard (livekit/node-sdks#700, livekit/node-sdks#698).
resourceFromAttributes, configure processors with spanProcessors, and pass
registerSpanProcessor when using a custom tracer provider.Include agent identity in telemetry resource attributes: stamp lk.agent_name when the job carries an agent name, and honor OTEL_RESOURCE_ATTRIBUTES (via the standard env detector) so environment-provided resource attributes reach cloud tracing. - #2110 (@jmcclanahan)
Add agent-simulation support: resolve the scenario dispatch from the job's lk.simulator.dispatch attribute into a SimulationContext, end the job when the simulator participant leaves, run the new defineAgent onSimulationEnd callback on finalizeSimulation, and disable STT/TTS/VAD and audio I/O under text simulations. - #2075 (@u9g)
Expose realtime provider response IDs on assistant message metrics as providerRequestIds. - #2084 (@rosetta-livekit-bot)
Fix a deadlock where a recorder-wrapped audio output could leave waitForPlayout stranded when - #2114 (@toubatbrian)
an interrupt arrived before the recorder had registered its segment. RecorderAudioOutput now
registers its own segment before forwarding a frame downstream, and attributes each playback
finish to the segment it belongs to instead of relying on a global counter.
A recorded segment is also timestamped when it opens rather than when the wrapped output accepts
its first frame, so a finish that lands while that frame is parked no longer clamps the segment's
playback position to zero and drop the audio the sink reported as played. And a segment whose
downstream capture throws now releases the capture latch on both the recorder and the wrapped
output, so a caller that retries after a transient rejection is no longer rejected forever with
recorder capture has no active segment.
A finish reported by the wrapped output settles its segment whether or not the recorder has been
flushed. The AudioOutput contract lets a sink report a finish as soon as its playout ends, and
TranscriptionSynchronizer does exactly that when it reconciles a dropped segment from
waitForPlayout, so requiring a flush first would strand the caller.
waitForPlayout no longer depends on a flush either. A segment the wrapped output never counted
is settled once that output reports its own playout complete, since at that point no finish can
ever arrive for it. Waiting for a flush instead only worked because performAudioForwarding — the
one thing that captures frames — happens to flush in a finally; a caller that waited without
flushing hung forever.
Behavior change: waitForPlayout now blocks while a frame is still in flight inside the wrapped
output. Previously it could return immediately with a fabricated
{ playbackPosition: 0, interrupted: false }, reporting a turn as completed while its audio had
not been handed to the sink yet. Callers that relied on the early return will now wait for the
real playback result.
Add Fish Audio model and option types to inference TTS. - #2102 (@rosetta-livekit-bot)
Support adaptive interruption gating for realtime models without server-side turn detection. - #2099 (@rosetta-livekit-bot)
Add inference.AvatarSession for provisioning avatar sessions through the LiveKit Inference gateway. - #2101 (@rosetta-livekit-bot)
Use required tool choice for message judge LLM calls. - #2128 (@rosetta-livekit-bot)
Allow failed inference avatar provisioning to be retried safely without duplicate provider sessions or cleanup callbacks, fall back to the connected room SID when dispatch metadata omits it, refresh gateway authentication across retries, and preserve the avatar audio output in RoomIO so synchronized transcription and avatar playback share the same output. - #2103 (@toubatbrian)
Do not drop realtime replies when the pre-reply chat context update times out. - #2083 (@rosetta-livekit-bot)
Fix updateAgent handoffs so run() captures onEnter output without waiting indefinitely on long-lived onEnter flows. - #2098 (@rosetta-livekit-bot)