Speech To Speech (STS)

Speech-to-Speech (STS), also known as Realtime in the SDK, is a single provider that handles listening, reasoning, and speaking over one connection. Realtime models include built-in STT, TTS, and turn detection — you configure llm=<provider>.Realtime() without separate speech plugins.

You still need edge, agent_user, instructions, API keys, and call lifecycle setup. See Voice Agents for the full guide.

How It Works

When the LLM is a Realtime model, the Agent selects RealtimeInferenceFlow:

• Call audio is sent to the Realtime API
• Model audio and transcripts are published back to the call
• STT, TTS, and turn detection plugins are automatically disabled

Quick Start

from vision_agents.core import Agent, User
from vision_agents.plugins import gemini, getstream

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You are a helpful voice assistant.",
    llm=gemini.Realtime(),  # no stt/tts/turn_detection needed
)

Realtime vs Custom Pipeline

Mode	Best for	You choose
Realtime	Fastest path, lowest latency	One provider for speech in and out
Custom Pipeline	Full control over each stage	STT, LLM, and TTS independently

Use a custom pipeline when you want to mix providers — for example Deepgram STT, Gemini LLM, and Inworld TTS.

Providers

Provider	Integration page
OpenAI	OpenAI Realtime
Gemini	Gemini Live
Qwen	Qwen OMNI
AWS Bedrock	AWS Bedrock
xAI	xAI
Inworld	Inworld

Next Steps

Voice Agents

Realtime setup and provider swapping

Interruption Handling

Barge-in and turn timing

Function Calling

Tools with realtime models

Custom Pipeline

Mix STT, LLM, and TTS providers