xAI (Grok)

xAI’s Grok provides advanced reasoning capabilities and real-time knowledge. The plugin supports conversation memory, streaming responses, and function calling (Grok 4.1+).

Vision Agents uses Stream Video for real-time WebRTC transport by default. External WebRTC transports are supported as well. Most AI providers offer free tiers to get started.

xAI also provides Realtime speech-to-speech and text-to-speech.

Installation

uv add "vision-agents[xai]"

Quick Start

from vision_agents.core import Agent, User
from vision_agents.plugins import xai, getstream, deepgram, elevenlabs

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You are a helpful assistant.",
    llm=xai.LLM(model="grok-4-latest"),
    stt=deepgram.STT(),
    tts=elevenlabs.TTS(),
)

Set XAI_API_KEY in your environment or pass api_key directly.

Parameters

Name	Type	Default	Description
model	str	"grok-4-latest"	Model ("grok-4-latest", "grok-4.1")
api_key	str	None	API key (defaults to XAI_API_KEY env var)
tools_max_rounds	int	3	Maximum tool-call rounds per response

Function Calling

Grok 4.1+ supports function calling:

@agent.llm.register_function(description="Get weather for a location")
async def get_weather(location: str) -> str:
    return f"The weather in {location} is sunny and 72°F"

See the Function Calling guide for details.

Events

The xAI plugin no longer documents provider-specific event classes for normal app code. Prefer core events and provider-agnostic response handling.

Next Steps

xAI Realtime

Speech-to-speech over WebSocket

xAI TTS

Text-to-speech with expressive voices