Refines default behavior for preemptive generation to better handle long or intermittent user speech, reducing unnecessary downstream inference and associated cost increases.
Also introduces PreemptiveGenerationOptions for developers who need fine-grained control over this behavior.
class PreemptiveGenerationOptions(TypedDict, total=False):
"""Configuration for preemptive generation."""
enabled: bool
"""Whether preemptive generation is enabled. Defaults to ``True``."""
preemptive_tts: bool
"""Whether to also run TTS preemptively before the turn is confirmed.
When ``False`` (default), only LLM runs preemptively; TTS starts once the
turn is confirmed and the speech is scheduled."""
max_speech_duration: float
"""Maximum user speech duration (s) for which preemptive generation
is attempted. Beyond this threshold, preemptive generation is skipped
since long utterances are more likely to change and users may expect
slower responses. Defaults to ``10.0``."""
max_retries: int
"""Maximum number of preemptive generation attempts per user turn.
The counter resets when the turn completes. Defaults to ``3``."""
Full Changelog: https://github.com/livekit/agents/compare/livekit-agents@1.5.3...livekit-agents@1.5.4
Fetched April 16, 2026