Twilio Phone Agent

Step-by-step guide to inbound and outbound phone calls with Twilio Media Streams

View Example on GitHub

Both outbound_phone_example.py and inbound_phone_and_rag_example.py live in this folder

Build inbound and outbound phone agents with Twilio Media Streams, Stream edge transport, and Gemini. This tutorial covers phone plumbing only — no RAG. For knowledge retrieval on calls, continue to Phone Support Agent (RAG).

Vision Agents uses Stream Video for real-time WebRTC transport by default. External WebRTC transports are supported as well. Most AI providers offer free tiers to get started.

What You Will Build

• Make an outbound call programmatically (e.g. call your cell to test audio)
• Answer inbound calls on your Twilio number with a voice AI agent
• Handle bidirectional audio via Twilio Media Streams over WebSocket
• Bridge phone audio into a Stream call with attach_phone_to_call

For optimal latency, deploy in US-east. Local development adds round-trip latency through ngrok and your machine.

Prerequisites

Create a .env file at the Vision Agents repo root:

STREAM_API_KEY=
STREAM_API_SECRET=
GOOGLE_API_KEY=
TWILIO_ACCOUNT_SID=
TWILIO_AUTH_TOKEN=

You also need ngrok and a Twilio phone number. See the Twilio integration for webhook and API details.

Run the example

1. Clone and install

   Clone the repo and install dependencies from the root:

   git clone [e-mail]:GetStream/Vision-Agents.git
   cd Vision-Agents
   uv sync
   
2. Start ngrok

   Expose port 8000 so Twilio can reach your local server:

   ngrok http 8000
   

   Copy the HTTPS hostname (without https://) — you’ll use it as NGROK_URL.

3. Configure your Twilio number

   In the Twilio Console:

   1. Go to Phone Numbers → Manage → Active numbers
   2. Select your number (or buy one)
   3. Under Voice Configuration, set A call comes in to Webhook
   4. Enter https://<NGROK_URL>/twilio/voice with method HTTP POST

   See Twilio integration for how the webhook handler works.

4. Make an outbound call

   In a new terminal, from the example directory:

   cd examples/03_phone_and_rag_example
   NGROK_URL=your-subdomain.ngrok-free.app uv run outbound_phone_example.py \
     --from [phone] \
     --to [phone]
   

   Replace with your Twilio number (--from) and a destination you can answer (--to, often your cell). This starts the HTTP server and initiates the outbound call.

5. Run the inbound agent

   With ngrok and your Twilio webhook still configured, start the inbound server:

   cd examples/03_phone_and_rag_example
   NGROK_URL=your-subdomain.ngrok-free.app uv run inbound_phone_and_rag_example.py
   

   RAG is optional at this stage — the agent runs with Gemini even without extra RAG configuration.

6. Call your number

   Dial your Twilio number from any phone. You should hear the AI agent answer and respond in real time.

How it works

Twilio uses TwiML to control calls. The voice webhook returns a <Connect><Stream> response that pipes audio to your WebSocket:

1. POST /twilio/voice — validates the Twilio signature, registers the call in TwilioCallRegistry, returns TwiML with a media stream URL
2. WS /twilio/media/{call_id}/{token} — accepts the WebSocket, runs TwilioMediaStream, validates the token
3. attach_phone_to_call — bridges Twilio mulaw audio ↔ the Stream call where your agent runs

The inbound script uses ProxyHeadersMiddleware so signature validation works when ngrok terminates HTTPS.

Next Steps

Phone Support Agent (RAG)

Add Gemini FileSearch or TurboPuffer knowledge retrieval

Twilio Integration

Plugin API reference and components

Phone Calling

Provider overview and learning path

Telnyx

Alternative telephony provider