Gemini LLM

Google’s Gemini provides powerful language models with built-in tools for search, code execution, RAG, and URL context. The LLM mode requires separate STT/TTS.

Vision Agents requires a Stream account for real-time transport. Most providers offer free tiers to get started.

Gemini also provides Realtime speech-to-speech with optional video over WebSocket.

Installation

uv add "vision-agents[gemini]"

Quick Start

from vision_agents.core import Agent, User
from vision_agents.plugins import gemini, getstream, deepgram, elevenlabs

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You are a helpful assistant.",
    llm=gemini.LLM(model="gemini-3.1-pro-preview"),
    stt=deepgram.STT(),
    tts=elevenlabs.TTS(),
)

Built-in Tools

Gemini provides built-in tools you can enable:

llm = gemini.LLM(
    model="gemini-3-flash-preview",
    tools=[
        gemini.tools.GoogleSearch(),
        gemini.tools.CodeExecution(),
        gemini.tools.FileSearch(store),  # RAG
        gemini.tools.URLContext(),
    ]
)

Tool	Description
GoogleSearch	Ground responses with web data
CodeExecution	Run Python code
FileSearch	RAG over your documents
URLContext	Read specific web pages

File Search (RAG)

Managed RAG with automatic chunking and retrieval:

from vision_agents.plugins import gemini

store = gemini.GeminiFilesearchRAG(name="my-knowledge-base")
await store.create()
await store.add_directory("./knowledge")

llm = gemini.LLM(
    model="gemini-3-flash-preview",
    tools=[gemini.tools.FileSearch(store)]
)

See the RAG guide for more details.

Function Calling

@agent.llm.register_function(description="Get weather for a location")
async def get_weather(location: str) -> dict:
    return {"temperature": "22°C", "condition": "Sunny"}

See the Function Calling guide for details.

Events

The Gemini plugin no longer documents provider-specific event classes for normal app code. Prefer core events and provider-agnostic response handling.

Next Steps

Gemini Realtime

Speech-to-speech with optional video

Build a Voice Agent

Get started with voice