Agents (Python)

Patch Changes

• add speech endpoint in azure ctor and azure speech sdk version upgrade - #2007 (@jayeshp19)

Patch Changes

• use 22.05khz by default for 11labs - a294d28c2af672a47f88f598f9fdb3fb13c39c38 (@davidzhao)

What's Changed

• fix audio resampler returning corrupted audio by @theomonnom https://github.com/livekit/rust-sdks/pull/627
• add websearch example & cleanup by @theomonnom in https://github.com/livekit/agents/pull/1960
• ignore stt transcription for realtime model by @longcw in https://github.com/livekit/agents/pull/1958
• add websearch example & cleanup by @theomonnom in https://github.com/livekit/agents/pull/1960
• export events by @theomonnom in https://github.com/livekit/agents/pull/1961
• add HTTP_PROXY for the worker connection by @theomonnom in https://github.com/livekit/agents/pull/1963
• remove error fields by @theomonnom in https://github.com/livekit/agents/pull/1964
• fix unserializable json on debug events by @theomonnom in https://github.com/livekit/agents/pull/1965
• Avoid session end deepgram errors by @zizhong in https://github.com/livekit/agents/pull/1967
• bump gemini sdk version & add video support for gemini multimodal by @jayeshp19 in https://github.com/livekit/agents/pull/1946
• make drop audio when uninterruptible optional by @longcw in https://github.com/livekit/agents/pull/1957
• add tool example with annotated arguments by @longcw in https://github.com/livekit/agents/pull/1969
• fix tool format when there is messages between tool call and output by @longcw in https://github.com/livekit/agents/pull/1977
• Remove english-only disclaimer from turn detector readme by @bcherry in https://github.com/livekit/agents/pull/1978
• fix: make STT fallback more robust by @zizhong in https://github.com/livekit/agents/pull/1947
• fix: gemini llm when tool choice set to required by @jayeshp19 in https://github.com/livekit/agents/pull/1984
• add chat_ctx.truncate by @longcw in https://github.com/livekit/agents/pull/1974
• support dynamic tools from llm_node by @longcw in https://github.com/livekit/agents/pull/1989
• add segment id to transcription text stream by @longcw in https://github.com/livekit/agents/pull/1951
• fix generate_tool_reply for realtime model by @longcw in https://github.com/livekit/agents/pull/1992
• Add new OpenAI models to ChatModels by @ZaneDash in https://github.com/livekit/agents/pull/2000
• fix looping for background audio using BuiltinAudioClip by @davidzhao in https://github.com/livekit/agents/pull/2001
• add with_azure in openai realtime by @jayeshp19 in https://github.com/livekit/agents/pull/1975
• add sample_rate for DataStreamAudioOutput by @longcw in https://github.com/livekit/agents/pull/2004
• fix OAI realtime model tool choice opt by @longcw in https://github.com/livekit/agents/pull/2005

New Contributors

• @simplegr33n made their first contribution in https://github.com/livekit/agents/pull/1639
• @Shubhrakanti made their first contribution in https://github.com/livekit/agents/pull/1747
• @ChenghaoMou made their first contribution in https://github.com/livekit/agents/pull/1775
• @paulwe made their first contribution in https://github.com/livekit/agents/pull/1864
• @ShayneP made their first contribution in https://github.com/livekit/agents/pull/1830
• @patrickscoleman made their first contribution in https://github.com/livekit/agents/pull/1920
• @Trivedi-grv made their first contribution in https://github.com/livekit/agents/pull/1884
• @msaelices made their first contribution in https://github.com/livekit/agents/pull/1923
• @zizhong made their first contribution in https://github.com/livekit/agents/pull/1967
• @ZaneDash made their first contribution in https://github.com/livekit/agents/pull/2000

Patch Changes

• google tts: configure api_endpoint based on location - #1890 (@jayeshp19)

Patch Changes

• added a multilingual turn detector option - #1736 (@jeradf)

Patch Changes

• release Resemble.ai TTS - #1833 (@davidzhao)

Patch Changes

• support multilingual with Nova-3 model - #1736 (@jeradf)

Patch Changes

• fix decoder: if no data was pushed, close the output channel - #1881 (@jayeshp19)

Patch Changes

• openai: default to use_realtime=False - #1783 (@davidzhao)

• fix(openai): pass NotGiven to OpenAI when instructions are omitted - #1834 (@davidzhao)

Patch Changes

• fixed thread safety in AudioStreamDecoder - #1736 (@jeradf)

• cleanup AudioStreamDecoder resources - #1736 (@jeradf)

Patch Changes

• fix: openai stt error when using detect language - #1755 (@jayeshp19)

Patch Changes

• expose turn_detection options with openai STT - #1726 (@davidzhao)

• feat(OpenAI STT): add support for semantic_vad - #1707 (@chasemcdo)

Patch Changes

• update to tts model and voices - #1725 (@davidzhao)

Patch Changes

• Added optional parameter to opt out from deepgrams model improvement plan - #1713 (@MatthiasGruba)

Minor Changes

• support for streaming STT, new STT/TTS models - #1701 (@davidzhao)

Patch Changes

• openai new STT model and voices - #1691 (@lundin)

• Make azure and openai take a timeout optionally. Also update the default timeout for Azure OpenAI to 5s from 10 minutes. - #1674 (@martin-purplefish)

Patch Changes

• initial version - #1689 (@davidzhao)

Patch Changes

• fix: double transcript issue for google stt - #1694 (@jayeshp19)

Patch Changes

• Remove unnecessary version pins - #1682 (@hauntsaninja)

• reduced retry interval to 2s - #1701 (@davidzhao)

Patch Changes

• Add string type support to model parameter - #1657 (@jayeshp19)

Patch Changes

• allow configurable api version in gemini realtime - #1656 (@jayeshp19)