Roboflow

Roboflow provides computer vision tools for object detection. The plugin offers both cloud-hosted inference (access to pre-trained models from Roboflow Universe) and local RF-DETR models.

Vision Agents uses Stream Video for real-time WebRTC transport by default. External WebRTC transports are supported as well. Most AI providers offer free tiers to get started.

Installation

uv add "vision-agents[roboflow]"

Cloud Detection

Uses Roboflow’s hosted API with pre-trained models.

from vision_agents.core import Agent, User
from vision_agents.plugins import roboflow, gemini, getstream

agent = Agent(
    edge=getstream.Edge(),
    agent_user=User(name="Assistant", id="agent"),
    instructions="You are a sports analyst.",
    llm=gemini.Realtime(fps=10),
    processors=[
        roboflow.RoboflowCloudDetectionProcessor(
            model_id="football-players-detection-3zvbc/20",
            classes=["player"],
            conf_threshold=0.5,
        )
    ],
)

Set ROBOFLOW_API_KEY in your environment or pass api_key directly.

Name	Type	Default	Description
model_id	str	—	Roboflow Universe model ID
classes	List[str]	None	Classes to detect (or all if None)
conf_threshold	float	0.5	Confidence threshold
fps	int	5	Frame processing rate
annotate	bool	True	Draw bounding boxes

Local Detection

Runs RF-DETR models locally without API calls.

processor = roboflow.RoboflowLocalDetectionProcessor(
    model_id="rfdetr-base",
    classes=["person"],
    conf_threshold=0.5,
)

Name	Type	Default	Description
model_id	str	"rfdetr-seg-preview"	RF-DETR model ("rfdetr-nano", "rfdetr-base", "rfdetr-large")
classes	List[str]	None	Classes to detect
conf_threshold	float	0.5	Confidence threshold

Cloud vs Local

	Cloud	Local
Use when	Access to Roboflow Universe models	Higher throughput, avoid rate limits
Pros	Thousands of pre-trained models, no GPU required	No API costs, lower latency, works offline
Cons	Requires API key, potential rate limits	Requires local compute, RF-DETR models only

Next Steps

Build a Voice Agent

Get started with voice

Build a Video Agent

Add video processing