Interruption Handling

Interruption handling ensures your voice agent responds naturally when users speak over the agent mid-response. Vision Agents handles interruptions automatically.

Vision Agents uses Stream Video for real-time WebRTC transport by default. External WebRTC transports are supported as well. Most AI providers offer free tiers to get started.

Automatic Handling

When you configure turn detection, the Agent class automatically:

1. Detects when the user starts speaking from STT/turn signals
2. Interrupts the active TTS or Realtime LLM
3. Discards stale audio from before the interruption
4. Flushes the audio track to clear buffered audio
5. Listens to the user’s new input
6. Responds appropriately

No custom event handlers required for basic interruption handling.

Stale audio protection

After an interruption, the agent ignores any already-generated audio that belongs to the previous response. This prevents “old” speech from leaking into the new turn.

Realtime APIs

If you’re using OpenAI Realtime, Gemini Live, AWS Bedrock, or Qwen, interruption handling is built-in at the model level. No turn detection plugin needed.

from vision_agents.core import Agent, User
from vision_agents.plugins import openai, getstream

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You're a helpful voice assistant.",
    llm=openai.Realtime(),  # Built-in interruption handling
)

Realtime APIs are recommended for the most natural conversation flow with minimal latency.

Traditional Pipeline Setup

For the STT → LLM → TTS pipeline, you need turn detection. Some STT plugins include it automatically:

from vision_agents.core import Agent, User
from vision_agents.plugins import openai, getstream, elevenlabs

# ElevenLabs STT has built-in VAD turn detection — no extra plugin needed
agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You're a helpful voice assistant.",
    llm=openai.LLM(model="gpt-5.4"),
    stt=elevenlabs.STT(),
    tts=elevenlabs.TTS(),
)

If your STT plugin does not include turn detection, add a separate plugin:

from vision_agents.core import Agent, User
from vision_agents.plugins import openai, getstream, deepgram, elevenlabs, smart_turn

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You're a helpful voice assistant.",
    llm=openai.LLM(model="gpt-5.4"),
    stt=deepgram.STT(),
    tts=elevenlabs.TTS(),
    turn_detection=smart_turn.TurnDetection(),
)

If you provide both an STT with built-in turn detection and a separate turn_detection plugin, the Agent automatically ignores the external plugin to prevent conflicts.

Custom Behavior

For app-triggered responses, make your own calls interruptible too:

await agent.say("One moment while I check that.", interrupt=True)
await agent.simple_response("Thanks for waiting.", interrupt=True)

You don’t need to call tts.interrupt() or llm.interrupt() yourself during normal conversation. The Agent handles interruptions automatically.

Tuning Sensitivity

Adjust turn detection parameters to control interruption response:

More Sensitive (Faster Response)

turn_detection = smart_turn.TurnDetection(
    buffer_in_seconds=0.5,
    confidence_threshold=0.3
)

Use when: You want immediate response to any sound. Trade-off: May trigger on background noise.

Less Sensitive (More Deliberate)

turn_detection = smart_turn.TurnDetection(
    buffer_in_seconds=2.0,
    confidence_threshold=0.7
)

Use when: You want to avoid false positives. Trade-off: Slower to respond to genuine interruptions.

Recommended Defaults

turn_detection = smart_turn.TurnDetection(
    buffer_in_seconds=1.5,
    confidence_threshold=0.5
)

Best Practices

Keep responses concise — Shorter responses mean fewer interruptions:

instructions="Keep responses under 2-3 sentences. Be concise."

Acknowledge interruptions — Add context in your instructions:

instructions="If interrupted, briefly acknowledge and address the user's new question."

Filter agent-originated signals — If you process raw turn signals, ignore events from the agent user:

if signal.participant and signal.participant.user_id == "agent":
    return

Troubleshooting

Issue	Solution
Agent doesn’t stop when interrupted	Verify turn_detection is configured; lower confidence_threshold
Agent stops too easily (false positives)	Increase confidence_threshold to 0.7; increase buffer_in_seconds to 2.0
Delay before responding to interruption	Decrease buffer_in_seconds to 0.5; consider Realtime API
Not working at all	Don’t use turn detection with Realtime LLMs (they handle it internally)

Next Steps

Turn Detection

VAD vs Turn Detection concepts

Smart Turn

Full parameter reference