73e5384c85ea9b29fa4c946f29c66bef80d5d160 (@davidzhao)You can now perform turn-by-turn evaluations on your agent interactions. Here's an example of how to validate expected behaviors:
result = await sess.run(user_input="Can I book an appointment? What's your availability for the next two weeks?")
result.expect.skip_next_event_if(type="message", role="assistant")
result.expect.next_event().is_function_call(name="list_available_slots")
result.expect.next_event().is_function_call_output()
await result.expect.next_event().is_message(role="assistant").judge(llm, intent="must confirm no availability")
Check out these practical examples: drive-thru, frontdesk
Documentation: https://docs.livekit.io/agents/build/testing/
This feature enables speculative initiation of LLM and TTS processing before the user's turn concludes, significantly reducing response latency by overlapping processing with user audio. Disabled by default:
session = AgentSession(..., preemptive_generation=True)
The end-of-turn model has been refined to reduce sensitivity to punctuation and better handle multilingual scenarios, notably improving Hindi language support. ' Documentation: https://docs.livekit.io/agents/build/turns/turn-detector/#supported-languages
Agent now supports tracing for LLM/TTS requests and user callbacks using OpenTelemetry. See LangFuse example for detailed implementation.
AgentTask is a new experimental subset feature allowing agents to terminate upon achieving specific goals. You can await AgentTasks directly in your workflows:
@function_tool
async def schedule_appointment(self, ctx: RunContext[Userdata], slot_id: str) -> str:
# Attempts to retrieve user email, allowing multiple agent-user interactions
email_result = await beta.workflows.GetEmailTask(chat_ctx=self.chat_ctx)
Combine Gemini or OpenAI's realtime STT/LLM with a separate TTS engine, optimizing your agent's voice interactions:
session = AgentSession(
llm=openai.realtime.RealtimeModel(modalities=["text"]),
# Alternatively: llm=google.beta.realtime.RealtimeModel(modalities=[Modality.TEXT]),
tts=openai.TTS(voice="ash"),
)
View the complete example.
Documentation: https://docs.livekit.io/agents/integrations/realtime/#separate-tts
Align transcripts accurately with speech outputs from TTS engines such as Cartesia and 11labs for improved synchronization:
session = AgentSession(..., use_tts_aligned_transcript=True)
Refer to the complete example.
Documentation: https://docs.livekit.io/agents/build/text/#tts-aligned-transcriptions
Transitioned to the Blingfire tokenization engine from the previous naive implementation, significantly enhancing handling and accuracy for multiple languages.
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.1.7...livekit-agents@1.2.0
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.1.6...livekit-agents@1.1.7
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.1.5...livekit-agents@1.1.6
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.1.4...livekit-agents@1.1.5
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.1.2...livekit-agents@1.1.4
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.1.0...livekit-agents@1.1.2
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.0.23...livekit-agents@1.1.0
with_letta OpenAI plugin by @mattzh72 in https://github.com/livekit/agents/pull/2182arcana model to allow for synthesis of long audio by @MaCaki in https://github.com/livekit/agents/pull/2343LLM.chat by @free-soellingeraj in https://github.com/livekit/agents/pull/2353Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.0.22...livekit-agents@1.0.23
32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)32e129ff1a4c3d28f363f4f2b2a355e29c8fe64d (@davidzhao)