Agents JS

fix(plugin-openai): honor OPENAI_BASE_URL for Realtime defaults. - #1650 (@rosetta-livekit-bot)

fix(plugin-openai): include OpenAI Realtime response status details in incomplete response logs. - #1631 (@rosetta-livekit-bot)

fix(llm): sort function tools to keep tool order invariant. - #1641 (@rosetta-livekit-bot)

feat(plugin-openai): stream input_audio_transcription.delta events on the OpenAI Realtime API as UserInputTranscribed partials (isFinal: false). Enables word-by-word user transcripts with gpt-realtime-whisper and any future delta-emitting transcription model. Accumulators are cleared on .completed, .failed, conversation.item.deleted, session close, and reconnect; .failed now emits a closing isFinal: true event when partials had streamed so consumers don't hang. - #1581 (@jjsquillante)

Updated dependencies [278af00740f8ade253485afc1d51a19382cb5a2c, 7c1cfcb1237f4aa7d36669754fc388bbaa872a71, 703cffbfda8a8b830b78f2c89e0b2284244dc120, 22b24ae18ca4f3dd690a948e9f9ab3ac9cdad4b0, 8524731b39097fe521952f83b210b999834bbeda, aa8d427391a79ec8e2ccda103e1460c720b73acf, 4207d54c037ad0206814ae5e246735e357bb4c3b, 106c2b6820751ce3802d77e1e1e74ece130efc89, 710d112f70ce1134f3098729cda6a950d1deb66c, 7479bd4640d2d82a4f0263dc0745e72bab733127, b157d241ffaf402c0a2b8dc05dbf7e9ed96d21ee, 181c868d75fe8828e5193fc530f454d52fc776e0, f470e5531da74d66484194125fc0f950b5a53431, 453b69965e0075f634f79c323194ceb16ccfe1a6, 0a27a700dcef89e6202872024d73452b47025788, 0f29f6bd34559ba98544c2c9621936b44a0573b8, c5eaadd27748aa2a69a4cb1a72b3de652fa841ef, 154189e4af2149c580aeaba7dfe3a8c79f3d682d, d85a30757405158fb3182be1c77aaec6acecdd67, ee078ec394e697a989bd0faf53aa043392fa7ae4, 7af2f8ebafe86fae52854ddc73b253f2d29da286, 33578844ba169f1485465ea96346567c449b7ea7, 62c8278eb1294088cad66285a799e84e20799eaf, 35a33ff0f89a23b17ea3ad79c642188e66e31604, ee078ec394e697a989bd0faf53aa043392fa7ae4, dfb34c27b8cb075ed77e63ceb5286fc504122abd, 98a46dd366fa69f9270d3ae780e50740ffe72c8a, 278af00740f8ade253485afc1d51a19382cb5a2c]:

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5
• @livekit/agents-plugin-openai@1.4.5

• @livekit/agents@1.4.5

Fix VAD speech buffer AudioFrame to include the prefix-padding pre-roll so START_OF_SPEECH / END_OF_SPEECH events deliver the full pre-rolled audio to downstream consumers (STT, transcription) and samplesPerChannel matches the data length. - #1636 (@chenghao-mou)

Reset active VAD streams on flush so STT end-of-speech can recover without recreating streams. STT end-of-speech now preserves the VAD-owned lastSpeakingTime instead of overwriting it, keeping the end-of-turn "no new speech" check reliable when VAD is active. - #1574 (@rosetta-livekit-bot)

Updated dependencies [278af00740f8ade253485afc1d51a19382cb5a2c, 7c1cfcb1237f4aa7d36669754fc388bbaa872a71, 703cffbfda8a8b830b78f2c89e0b2284244dc120, 22b24ae18ca4f3dd690a948e9f9ab3ac9cdad4b0, 8524731b39097fe521952f83b210b999834bbeda, aa8d427391a79ec8e2ccda103e1460c720b73acf, 4207d54c037ad0206814ae5e246735e357bb4c3b, 106c2b6820751ce3802d77e1e1e74ece130efc89, 710d112f70ce1134f3098729cda6a950d1deb66c, 7479bd4640d2d82a4f0263dc0745e72bab733127, b157d241ffaf402c0a2b8dc05dbf7e9ed96d21ee, 181c868d75fe8828e5193fc530f454d52fc776e0, f470e5531da74d66484194125fc0f950b5a53431, 453b69965e0075f634f79c323194ceb16ccfe1a6, 0a27a700dcef89e6202872024d73452b47025788, 0f29f6bd34559ba98544c2c9621936b44a0573b8, c5eaadd27748aa2a69a4cb1a72b3de652fa841ef, 154189e4af2149c580aeaba7dfe3a8c79f3d682d, d85a30757405158fb3182be1c77aaec6acecdd67, ee078ec394e697a989bd0faf53aa043392fa7ae4, 7af2f8ebafe86fae52854ddc73b253f2d29da286, 33578844ba169f1485465ea96346567c449b7ea7, 62c8278eb1294088cad66285a799e84e20799eaf, 35a33ff0f89a23b17ea3ad79c642188e66e31604, ee078ec394e697a989bd0faf53aa043392fa7ae4, dfb34c27b8cb075ed77e63ceb5286fc504122abd, 98a46dd366fa69f9270d3ae780e50740ffe72c8a, 278af00740f8ade253485afc1d51a19382cb5a2c]:

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5
• @livekit/agents-plugin-mistralai@1.4.5

fix(llm): sort function tools to keep tool order invariant. - #1641 (@rosetta-livekit-bot)

Updated dependencies [278af00740f8ade253485afc1d51a19382cb5a2c, 7c1cfcb1237f4aa7d36669754fc388bbaa872a71, 703cffbfda8a8b830b78f2c89e0b2284244dc120, 22b24ae18ca4f3dd690a948e9f9ab3ac9cdad4b0, 8524731b39097fe521952f83b210b999834bbeda, aa8d427391a79ec8e2ccda103e1460c720b73acf, 4207d54c037ad0206814ae5e246735e357bb4c3b, 106c2b6820751ce3802d77e1e1e74ece130efc89, 710d112f70ce1134f3098729cda6a950d1deb66c, 7479bd4640d2d82a4f0263dc0745e72bab733127, b157d241ffaf402c0a2b8dc05dbf7e9ed96d21ee, 181c868d75fe8828e5193fc530f454d52fc776e0, f470e5531da74d66484194125fc0f950b5a53431, 453b69965e0075f634f79c323194ceb16ccfe1a6, 0a27a700dcef89e6202872024d73452b47025788, 0f29f6bd34559ba98544c2c9621936b44a0573b8, c5eaadd27748aa2a69a4cb1a72b3de652fa841ef, 390da2c89aa03eacd2c48b7720d966f75fbd5387, 154189e4af2149c580aeaba7dfe3a8c79f3d682d, d85a30757405158fb3182be1c77aaec6acecdd67, ee078ec394e697a989bd0faf53aa043392fa7ae4, 7af2f8ebafe86fae52854ddc73b253f2d29da286, 33578844ba169f1485465ea96346567c449b7ea7, 62c8278eb1294088cad66285a799e84e20799eaf, 35a33ff0f89a23b17ea3ad79c642188e66e31604, ee078ec394e697a989bd0faf53aa043392fa7ae4, dfb34c27b8cb075ed77e63ceb5286fc504122abd, 98a46dd366fa69f9270d3ae780e50740ffe72c8a, 278af00740f8ade253485afc1d51a19382cb5a2c]:

• @livekit/agents@1.4.5
• @livekit/agents-plugin-silero@1.4.5

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

Add Soniox STT support and surface per-run source and target language segments on STT speech data. - #1602 (@rosetta-livekit-bot)

Surface Soniox STT server error frames as API status errors. - #1602 (@rosetta-livekit-bot)

Updated dependencies [278af00740f8ade253485afc1d51a19382cb5a2c, 7c1cfcb1237f4aa7d36669754fc388bbaa872a71, 703cffbfda8a8b830b78f2c89e0b2284244dc120, 22b24ae18ca4f3dd690a948e9f9ab3ac9cdad4b0, 8524731b39097fe521952f83b210b999834bbeda, aa8d427391a79ec8e2ccda103e1460c720b73acf, 4207d54c037ad0206814ae5e246735e357bb4c3b, 106c2b6820751ce3802d77e1e1e74ece130efc89, 710d112f70ce1134f3098729cda6a950d1deb66c, 7479bd4640d2d82a4f0263dc0745e72bab733127, b157d241ffaf402c0a2b8dc05dbf7e9ed96d21ee, 181c868d75fe8828e5193fc530f454d52fc776e0, f470e5531da74d66484194125fc0f950b5a53431, 453b69965e0075f634f79c323194ceb16ccfe1a6, 0a27a700dcef89e6202872024d73452b47025788, 0f29f6bd34559ba98544c2c9621936b44a0573b8, c5eaadd27748aa2a69a4cb1a72b3de652fa841ef, 154189e4af2149c580aeaba7dfe3a8c79f3d682d, d85a30757405158fb3182be1c77aaec6acecdd67, ee078ec394e697a989bd0faf53aa043392fa7ae4, 7af2f8ebafe86fae52854ddc73b253f2d29da286, 33578844ba169f1485465ea96346567c449b7ea7, 62c8278eb1294088cad66285a799e84e20799eaf, 35a33ff0f89a23b17ea3ad79c642188e66e31604, ee078ec394e697a989bd0faf53aa043392fa7ae4, dfb34c27b8cb075ed77e63ceb5286fc504122abd, 98a46dd366fa69f9270d3ae780e50740ffe72c8a, 278af00740f8ade253485afc1d51a19382cb5a2c]:

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

fix(sarvam): thread language_probability into STT confidence - #1591 (@rosetta-livekit-bot)

Updated dependencies [278af00740f8ade253485afc1d51a19382cb5a2c, 7c1cfcb1237f4aa7d36669754fc388bbaa872a71, 703cffbfda8a8b830b78f2c89e0b2284244dc120, 22b24ae18ca4f3dd690a948e9f9ab3ac9cdad4b0, 8524731b39097fe521952f83b210b999834bbeda, aa8d427391a79ec8e2ccda103e1460c720b73acf, 4207d54c037ad0206814ae5e246735e357bb4c3b, 106c2b6820751ce3802d77e1e1e74ece130efc89, 710d112f70ce1134f3098729cda6a950d1deb66c, 7479bd4640d2d82a4f0263dc0745e72bab733127, b157d241ffaf402c0a2b8dc05dbf7e9ed96d21ee, 181c868d75fe8828e5193fc530f454d52fc776e0, f470e5531da74d66484194125fc0f950b5a53431, 453b69965e0075f634f79c323194ceb16ccfe1a6, 0a27a700dcef89e6202872024d73452b47025788, 0f29f6bd34559ba98544c2c9621936b44a0573b8, c5eaadd27748aa2a69a4cb1a72b3de652fa841ef, 154189e4af2149c580aeaba7dfe3a8c79f3d682d, d85a30757405158fb3182be1c77aaec6acecdd67, ee078ec394e697a989bd0faf53aa043392fa7ae4, 7af2f8ebafe86fae52854ddc73b253f2d29da286, 33578844ba169f1485465ea96346567c449b7ea7, 62c8278eb1294088cad66285a799e84e20799eaf, 35a33ff0f89a23b17ea3ad79c642188e66e31604, ee078ec394e697a989bd0faf53aa043392fa7ae4, dfb34c27b8cb075ed77e63ceb5286fc504122abd, 98a46dd366fa69f9270d3ae780e50740ffe72c8a, 278af00740f8ade253485afc1d51a19382cb5a2c]:

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5

feat(cartesia): add Ink-2 STT and update example to use Ink-2 + Sonic-3.5 - #1587 (@charlotte-zhuang)

fix(cartesia): surface TTS websocket server errors - #1534 (@bowdens)

Cartesia error frames over the synthesis WebSocket are now classified by status_code: 5xx raises a retryable APIStatusError (carrying the status code) so the base SynthesizeStream retries and emits tts_error once retries are exhausted, while 4xx (e.g. empty-transcript on function-call turns) is logged at debug and the segment finishes cleanly.

Updated dependencies [278af00740f8ade253485afc1d51a19382cb5a2c, 7c1cfcb1237f4aa7d36669754fc388bbaa872a71, 703cffbfda8a8b830b78f2c89e0b2284244dc120, 22b24ae18ca4f3dd690a948e9f9ab3ac9cdad4b0, 8524731b39097fe521952f83b210b999834bbeda, aa8d427391a79ec8e2ccda103e1460c720b73acf, 4207d54c037ad0206814ae5e246735e357bb4c3b, 106c2b6820751ce3802d77e1e1e74ece130efc89, 710d112f70ce1134f3098729cda6a950d1deb66c, 7479bd4640d2d82a4f0263dc0745e72bab733127, b157d241ffaf402c0a2b8dc05dbf7e9ed96d21ee, 181c868d75fe8828e5193fc530f454d52fc776e0, f470e5531da74d66484194125fc0f950b5a53431, 453b69965e0075f634f79c323194ceb16ccfe1a6, 0a27a700dcef89e6202872024d73452b47025788, 0f29f6bd34559ba98544c2c9621936b44a0573b8, c5eaadd27748aa2a69a4cb1a72b3de652fa841ef, 154189e4af2149c580aeaba7dfe3a8c79f3d682d, d85a30757405158fb3182be1c77aaec6acecdd67, ee078ec394e697a989bd0faf53aa043392fa7ae4, 7af2f8ebafe86fae52854ddc73b253f2d29da286, 33578844ba169f1485465ea96346567c449b7ea7, 62c8278eb1294088cad66285a799e84e20799eaf, 35a33ff0f89a23b17ea3ad79c642188e66e31604, ee078ec394e697a989bd0faf53aa043392fa7ae4, dfb34c27b8cb075ed77e63ceb5286fc504122abd, 98a46dd366fa69f9270d3ae780e50740ffe72c8a, 278af00740f8ade253485afc1d51a19382cb5a2c]:

• @livekit/agents@1.4.5

• @livekit/agents@1.4.5
• @livekit/agents-plugin-openai@1.4.5

• @livekit/agents@1.4.5