ElevenAgents | ElevenLabs Documentation

Agents accomplish tasks through natural dialogue - from quick requests to complex, open-ended workflows. ElevenLabs provides voice-rich, expressive models, developer tools for building multimodal agents, and tools to monitor and evaluate agent performance at scale.

Configure

Configure multimodal agents with our developer toolkit, dashboard, or visual workflow builder

Deploy

Integrate multimodal agents across telephony systems, web, and mobile

Monitor

Evaluate agent performance with built-in testing, evals, and analytics

Platform capabilities

From design to deployment to optimization, ElevenLabs provides everything you need to build agents at scale.

Design and configure

Goal	Guide	Description
Create conversation workflows	Workflows	Build multi-step workflows with visual workflow builder
Write system prompts	System prompt	Learn best practices for crafting effective agent prompts
Select language model	Models	Choose from supported LLMs or bring your own custom model
Control conversation flow	Conversation flow	Configure turn-taking, interruptions, and timeout settings
Configure voice & language	Voice & language	Select from 10,000+ voices across 31 languages with customization options
Add knowledge to agent	Knowledge base	Upload documents and enable RAG for grounded responses
Connect tools	Tools	Enable agents to call clients & APIs to perform actions
Personalize each conversation	Personalization	Use dynamic variables and overrides for per-conversation customization
Secure agent access	Authentication	Implement custom authentication for protected agent access

Connect and deploy

Goal	Guide	Description
Build with React components	ElevenLabs UI	Pre-built components library for audio & agent apps (shadcn-based)
Embed widget in website	Widget	Add a customizable web widget to any website
Build React web apps	React SDK	Voice-enabled React hooks and components
Build iOS apps	Swift SDK	Native iOS SDK for voice agents
Build Android apps	Kotlin SDK	Native Android SDK for voice agents
Build React Native apps	React Native SDK	Cross-platform iOS and Android with React Native
Connect via SIP trunk	SIP trunk	Integrate with existing telephony infrastructure
Make batch outbound calls	Batch calls	Trigger multiple calls programmatically
Use Twilio integration	Twilio	Native Twilio integration for phone calls
Build custom integrations	WebSocket API	Low-level WebSocket protocol for custom implementations
Receive real-time events	Events	Subscribe to conversation events and updates

Monitor and optimize

Goal	Guide	Description
List users by external ID	Users	See end users and open their conversations
Search transcripts	Searching conversations	Keyword and semantic search in Conversation history
Run A/B tests	Experiments	Test agent configuration changes with live traffic
Test agent behavior	Testing	Create and run automated tests for your agents
Analyze conversation quality	Conversation analysis	Extract insights and evaluate conversation outcomes
Track metrics & analytics	Analytics	Monitor performance metrics and conversation history
Configure data retention	Privacy	Set retention policies for conversations and audio
Reduce LLM costs	Cost optimization	Monitor and optimize language model expenses
Manage agents as code	CLI	Version control and deploy agents from your terminal
Manage agents from Claude	Hosted MCP	Create and update agents from any MCP client

Architecture

ElevenAgents coordinates 4 core components:

A fine-tuned Speech to Text (ASR) model for speech recognition
Your choice of language model or custom LLM
A low-latency Text to Speech (TTS) model across 10,000+ voices and 70+ languages
A proprietary turn-taking model that handles conversation timing

Voice agents can also use a Realtime model, an ElevenLabs model built for low-latency voice conversation.

How Eleven Realtime relates to ElevenAgents and Speech Engine

ElevenLabs offers three ways to add voice to a product, each suited to a different level of control and integration effort.

Product	What it is	Primary audience
ElevenAgents	A full omnichannel agents platform. Runs on Realtime models, or on a fully cascaded, configurable pipeline with custom TTS, LLM, STT, workflows, evals, and monitoring.	Developers and enterprises building production agents at scale.
Realtime models	A standalone API exposing a series of speech-to-speech models, each optimized for a different use case. Available directly through the Realtime API, or as a model option inside ElevenAgents.	Developers and enterprises who want to add voice with minimal complexity, including teams already using another realtime voice API and looking to switch.
Speech Engine	The speech orchestration layer on its own, for teams that have already built a text-based chat agent and want to add voice.	Developers and enterprises who have already built a text agent and want to add voice.

See the OpenAI Realtime API compatibility guide for connecting to Realtime models using OpenAI’s Realtime SDK and wire protocol, and the Speech Engine overview for adding voice to an existing text agent.

Quickstart

Build your first agent in 5 minutes