Model Context Protocol (MCP)

The Model Context Protocol (MCP) is an open standard that defines how LLMs interact with external tools and services. Vision Agents supports MCP for function calling and connecting to external tool servers.

How Vision Agents Uses MCP

Vision Agents provides two ways to give your agent access to tools:

1. Function Registration — Decorate Python functions to make them callable by the LLM
2. MCP Servers — Connect to local or remote servers that expose tools

Both approaches work with any LLM that supports function calling (OpenAI, Gemini, etc.) and with Realtime models.

Function Registration

Use @llm.register_function() to make any Python function available to your agent:

from vision_agents.plugins import openai

llm = openai.LLM(model="gpt-5.4")

@llm.register_function(description="Get current weather for a location")
async def get_weather(location: str) -> dict:
    return {"location": location, "temperature": "22°C", "condition": "Sunny"}

# The LLM will automatically call get_weather() when relevant
response = await llm.simple_response("What's the weather like in London?")

MCP Servers

Connect to MCP servers for access to external services:

from vision_agents.core.mcp import MCPServerRemote, MCPServerLocal
from vision_agents.core.agents import Agent

# Remote server (HTTP)
github_server = MCPServerRemote(
    url="https://api.githubcopilot.com/mcp/",
    headers={"Authorization": f"Bearer {token}"}
)

# Local server (stdio)
local_server = MCPServerLocal(command="uv run my_mcp_server.py")

# Connect to your agent
agent = Agent(
    edge=getstream.Edge(),
    llm=openai.LLM(model="gpt-5.4"),
    agent_user=User(name="Assistant", id="agent"),
    mcp_servers=[github_server, local_server]
)

When you add MCP servers to an agent, tools are automatically discovered and registered with the LLM.

When to Use Each Approach

Approach	Best For
Function Registration	Simple tools, inline logic, prototyping
MCP Servers	External APIs, shared tooling, complex integrations

You can combine both — register local functions while also connecting MCP servers.

For detailed examples including parameter types, custom function names, and real-world patterns, see the MCP and Function Calling Guide.