GET STARTED

Changelog

March 17, 2025

Conversational AI

  • Default LLM update: Changed the default agent LLM from Gemini 1.5 Flash to Gemini 2.0 Flash for improved performance.
  • Fixed incorrect conversation abandons: Improved detection of conversation continuations, preventing premature abandons when users repeat themselves.
  • Twilio information in history: Added Twilio call details to conversation history for better tracking.
  • Knowledge base redesign: Redesigned the knowledge base interface.
  • System dynamic variables: Added system dynamic variables to use time, conversation id, caller id and other system values as dynamic variables in prompts and tools.
  • Twilio client initialisation: Adds an agent level override for conversation initiation client data twilio webhook.
  • RAG chunks in history: Added retrieved chunks by RAG to the call transcripts in the history view.

Speech to Text

  • Reduced pricing: Reduced the pricing of our Scribe model, see more here.
  • Improved VAD detection: Enhanced Voice Activity Detection with better pause detection at segment boundaries and improved handling of silent segments.
  • Enhanced diarization: Improved speaker clustering with a better ECAPA model, symmetric connectivity matrix, and more selective speaker embedding generation.
  • Fixed ASR bugs: Resolved issues with VAD rounding, silence and clustering that affected transcription accuracy.

Studio

  • Disable publishing UI: Added ability to disable the publishing interface for specific workspace members to support enterprise workflows.
  • Snapshot API improvement: Modified endpoints for project and chapter snapshots to return an empty list instead of throwing errors when snapshots can’t be downloaded.
  • Disabled auto-moderation: Turned off automatic moderation based on Text to Speech generations in Studio.

Workspaces

  • Fixed API key editing: Resolved an issue where editing workspace API keys would reset character limits to zero, causing the keys to stop working.
  • Optimized free subscriptions: Fixed an issue with refreshing free subscription character limits,

API

New Endpoints

Updated Endpoints

Dubbing

Project Management

  • Updated Project endpoints:
    • Add project - Made metadata, project_name, description nullable
    • Create podcast - Made title, description, author nullable
    • Get project - Made last_modified_at, created_at, project_name nullable
    • Add chapter - Made chapter_id, word_count, statistics nullable
    • Update chapter - Made content and blocks properties nullable

Conversational AI

  • Updated Conversational AI endpoints:
    • Update agent - Made conversation_config, platform_settings nullable and added workspace_overrides property
    • Create agent - Made agent_name, prompt, widget_config nullable and added workspace_overrides property
    • Add to knowledge base - Made document_name nullable
    • Get conversation - Added twilio_call_data model and made transcript, metadata nullable

Text to Speech

Voice Management

  • Updated Voice endpoints:

Speech to Text

Other Updates

March 10, 2025

Conversational AI

  • HIPAA compliance: Conversational AI is now HIPAA compliant on appropriate plans, when a BAA is signed, zero-retention mode is enabled and appropriate LLMs are used. For access please contact sales
  • Cascade LLM: Added dynamic dispatch during the LLM step to other LLMs if your default LLM fails. This results in higher latency but prevents the turn failing.
  • Better error messages: Added better error messages for websocket failures.
  • Audio toggling: Added ability to select only user or agent audio in the conversation playback.

Scribe

  • HIPAA compliance: Added a zero retention mode to Scribe to be HIPAA compliant.
  • Diarization: Increased time length of audio files that can be transcribed with diarization from 8 minutes to 2 hours.
  • Cheaper pricing: Updated Scribe’s pricing to be cheaper, as low as $0.22 per hour for the Business tier.
  • Memory usage: Shipped improvements to Scribe’s memory usage.
  • Fixed timestamps: Fixed an issue that was causing incorrect timestamps to be returned.
  • Biased keywords: Added biased keywords to improve Scribe’s performance.

Text to Speech

  • Pronunciation dictionaries: Fixed pronunciation dictionary rule application for replacements that contain symbols.

Dubbing

  • Studio support: Added support for creating dubs with dubbing_studio enabled, allowing for more advanced dubbing workflows beyond one-off dubs.

Voices

  • Verification: Fixed an issue where users on probation could not verify their voice clone.

API

New Endpoints

Updated Endpoints

Studio Projects

Voice Management

  • Updated Voice endpoints with several property changes:

Conversational AI

  • Updated Conversational AI agent endpoints:
    • Update agent - Modified conversation_config, agent, platform_settings, and widget properties
    • Create agent - Modified conversation_config, agent, prompt, platform_settings, widget properties and added shareable_page_show_terms
    • Get agent - Modified conversation_config, agent, platform_settings, and widget properties
    • Get widget - Modified widget_config property and added shareable_page_show_terms

Knowledge Base

Other Updates

Removed Endpoints

  • Temporarily removed Conversational AI tools endpoints:

    • Get tool
    • List tools
    • Update tool
    • Create tool
    • Delete tool
March 3, 2025

Dubbing

  • Scribe for speech recognition: Dubbing Studio now uses Scribe by default for speech recognition to improve accuracy.

Speech to Text

  • Fixes: Shipped several fixes improving the stability of Speech to Text.

Conversational AI

  • Speed control: Added speed control to an agent’s settings in Conversational AI.
  • Post call webhook: Added the option of sending post-call webhooks after conversations are completed.
  • Improved error messages: Added better error messages to the Conversational AI websocket.
  • Claude 3.7 Sonnet: Added Claude 3.7 Sonnet as a new LLM option in Conversational AI.

API

New Endpoints

Updated Endpoints

  • Added prompt_injectable property to knowledge base endpoints
  • Added name property to Knowledge Base document creation and retrieval endpoints:
  • Added speed property to agent creation
  • Removed secrets property from agent endpoints (now handled by dedicated secrets endpoints)
  • Added secret deletion endpoint for removing secrets
  • Removed secrets property from settings endpoints
February 25, 2025

Speech to Text

  • ElevenLabs launched a new state of the art Speech to Text API available in 99 languages.

Text to Speech

  • Speed control: Added speed control to the Text to Speech API.

Studio

  • Auto-assigned projects: Increased token limits for auto-assigned projects from 1 month to 3 months worth of tokens, addressing user feedback about working on longer projects.
  • Language detection: Added automatic language detection when generating audio for the first time, with suggestions to switch to Eleven Turbo v2.5 for languages not supported by Multilingual v2 (Hungarian, Norwegian, Vietnamese).
  • Project export: Enhanced project exporting in ElevenReader with better metadata tracking.

Dubbing

  • Clip overlap prevention: Added automatic trimming of overlapping clips in dubbing jobs to ensure clean audio tracks for each speaker and language.

Voice Management

  • Instant Voice Cloning: Improved preview generation for Instant Voice Cloning v2, making previews available immediately.

Conversational AI

  • Agent ownership: Added display of agent creators in the agent list, improving visibility and management of shared agents.

Web app

  • Dark mode: Added dark mode to the web app.

API

February 17, 2025

Conversational AI

  • Tool calling fix: Fixed an issue where tool calling was not working with agents using gpt-4o mini. This was due to a breaking change in the OpenAI API.
  • Tool calling improvements: Added support for tool calling with dynamic variables inside objects and arrays.
  • Dynamic variables: Fixed an issue where dynamic variables of a conversation were not being displayed correctly.

Voice Isolator

  • Fixed: Fixed an issue that caused the voice isolator to not work correctly temporarily.

Workspace

  • Billing: Improved billing visibility by differentiating rollover, cycle, gifted, and usage-based credits.
  • Usage Analytics: Improved usage analytics load times and readability.
  • Fine grained fiat billing: Added support for customizable pricing based on several factors.

API

  • Added phone_numbers property to Agent responses
  • Added usage metrics to subscription_extras in User endpoint:
    • unused_characters_rolled_over_from_previous_period
    • overused_characters_rolled_over_from_previous_period
    • usage statistics
  • Added enable_conversation_initiation_client_data_from_webhook to Agent creation
  • Updated Agent endpoints with consolidated settings for:
    • platform_settings
    • overrides
    • safety
  • Deprecated with_settings parameter in Voice retrieval endpoint
February 10, 2025

Conversational AI

Studio

  • GenFM: Updated the create podcast endpoint to accept multiple input sources.
  • GenFM: Fixed an issue where GenFM was creating empty podcasts.

Enterprise

  • New workspace group endpoints: Added new endpoints to manage workspace groups.

API

Studio (formerly Projects)

All /v1/projects/* endpoints have been deprecated in favor of the new /v1/studio/projects/* endpoints. The following endpoints are now deprecated:

  • All operations on /v1/projects/
  • All operations related to chapters, snapshots, and content under /v1/projects/*

Conversational AI

  • POST /v1/convai/add-tool - Use POST /v1/convai/tools instead
  • DELETE /v1/convai/agents/{agent_id} - Response type is no longer an object
  • GET /v1/convai/tools - Response type changed from array to object with a tools property

Conversational AI Updates

  • GET /v1/convai/agents/{agent_id} - Updated conversation configuration and agent properties
  • PATCH /v1/convai/agents/{agent_id} - Added use_tool_ids parameter for tool management
  • POST /v1/convai/agents/create - Added tool integration via use_tool_ids

Knowledge Base & Tools

  • GET /v1/convai/agents/{agent_id}/knowledge-base/{documentation_id} - Added name and access_level properties
  • GET /v1/convai/knowledge-base/{documentation_id} - Added name and access_level properties
  • GET /v1/convai/tools/{tool_id} - Added dependent_agents property
  • PATCH /v1/convai/tools/{tool_id} - Added dependent_agents property

GenFM

  • POST /v1/projects/podcast/create - Added support for multiple input sources

Studio (formerly Projects)

New endpoints replacing the deprecated /v1/projects/* endpoints

  • GET /v1/studio/projects: List all projects
  • POST /v1/studio/projects: Create a project
  • GET /v1/studio/projects/{project_id}: Get project details
  • DELETE /v1/studio/projects/{project_id}: Delete a project

Knowledge Base Management

  • GET /v1/convai/knowledge-base: List all knowledge base documents
  • DELETE /v1/convai/knowledge-base/{documentation_id}: Delete a knowledge base
  • GET /v1/convai/knowledge-base/{documentation_id}/dependent-agents: List agents using this knowledge base

Workspace Groups - New enterprise features for team management

  • GET /v1/workspace/groups/search: Search workspace groups
  • POST /v1/workspace/groups/{group_id}/members: Add members to a group
  • POST /v1/workspace/groups/{group_id}/members/remove: Remove members from a group

Tools

  • POST /v1/convai/tools: Create new tools for agents

Socials

  • ElevenLabs Developers: Follow our new developers account on X @ElevenLabsDevs
February 4, 2025

Conversational AI

  • Agent monitoring: Added a new dashboard for monitoring conversational AI agents’ activity. Check out your’s here.
  • Proactive conversations: Enhanced capabilities with improved timeout retry logic. Learn more
  • Tool calls: Fixed timeout issues occurring during tool calls
  • Allowlist: Fixed implementation of allowlist functionality.
  • Content summarization: Added Gemini as a fallback model to ensure service reliability
  • Widget stability: Fixed issue with dynamic variables causing the Conversational AI widget to fail

Reader

  • Trending content: Added carousel showcasing popular articles and trending content
  • New publications: Introduced dedicated section for recent ElevenReader Publishing releases

Studio (formerly Projects)

  • Projects is now Studio and is now generally available to everyone
  • Chapter content editing: Added support for editing chapter content through the public API, enabling programmatic updates to chapter text and metadata
  • GenFM public API: Added public API support for podcast creation through GenFM. Key features include:
    • Conversation mode with configurable host and guest voices
    • URL-based content sourcing
    • Customizable duration and highlights
    • Webhook callbacks for status updates
    • Project snapshot IDs for audio downloads

SDKs

  • Swift: fixed an issue where resources were not being released after the end of a session
  • Python: added uv support
  • Python: fixed an issue where calls were not ending correctly

API

  • Added POST v1/workspace/invites/add-bulk endpoint to enable inviting multiple users simultaneously
  • Added POST v1/projects/podcast/create endpoint for programmatic podcast generation through GenFM
  • Added ‘v1/convai/knowledge-base/:documentation_id’ endpoints with CRUD operations for Conversational AI
  • Added PATCH v1/projects/:project_id/chapters/:chapter_id endpoint for updating project chapter content and metadata
  • Added group_ids parameter to Workspace Invite endpoint for group-based access control
  • Added structured content property to Chapter response objects
  • Added retention_days and delete_transcript_and_pii data retention parameters to Agent creation
  • Added structured response to AudioNative content
  • Added convai_chars_per_minute usage metric to User endpoint
  • Added media_metadata field to Dubbing response objects
  • Added GDPR-compliant deletion_settings to Conversation responses
  • Deprecated Knowledge Base legacy endpoints:
    • POST /v1/convai/agents/{agent_id}/add-to-knowledge-base
    • GET /v1/convai/agents/{agent_id}/knowledge-base/{documentation_id}
  • Updated Agent endpoints with consolidated privacy control parameters
January 27, 2025

Docs

  • Shipped our new docs: we’re keen to hear your thoughts, you can reach out by opening an issue on GitHub or chatting with us on Discord

Conversational AI

  • Dynamic variables: Available in the dashboard and SDKs. Learn more
  • Interruption handling: Now possible to ignore user interruptions in Conversational AI. Learn more
  • Twilio integration: Shipped changes to increase audio quality when integrating with Twilio
  • Latency optimization: Published detailed blog post on latency optimizations. Read more
  • PCM 8000: Added support for PCM 8000 to Conversational AI agents
  • Websocket improvements: Fixed unexpected websocket closures

Projects

  • Auto-regenerate: Auto-regeneration now available by default at no extra cost
  • Content management: Added updateContent method for dynamic content updates
  • Audio conversion: New auto-convert and auto-publish flags for seamless workflows

API

  • Added Update Project endpoint for project editing
  • Added Update Content endpoint for AudioNative content management
  • Deprecated quality_check_on parameter in project operations. It is now enabled for all users at no extra cost
  • Added apply_text_normalization parameter to project creation with modes ‘auto’, ‘on’, ‘apply_english’ and ‘off’ for controlling text normalization during project creation
  • Added alpha feature auto_assign_voices in project creation to automatically assign voices to phrases
  • Added auto_convert flag to project creation to automatically convert projects to audio
  • Added support for creating Conversational AI agents with dynamic variables
  • Added voice_slots_used to Subscription model to track number of custom voices used in a workspace to the User endpoint
  • Added user_id field to User endpoint
  • Marked legacy AudioNative creation parameters (image, small, sessionization) as deprecated parameters
  • Agents platform now supports call_limits containing either agent_concurrency_limit or daily_limit or both parameters to control simultaneous and daily conversation limits for agents
  • Added support for language_presets in conversation_config to customize language-specific settings

SDKs

  • Cross-Runtime Support: Now compatible with Bun 1.1.45+ and Deno 2.1.7+
  • Regenerated SDKs: We regenerated our SDKs to be up to date with the latest API spec. Check out the latest Python SDK release and JS SDK release
  • Dynamic Variables: Fixed an issue where dynamic variables were not being handled correctly, they are now correctly handled in all SDKs
January 16, 2025

Product

Conversational AI

  • Additional languages: Add a language dropdown to your widget so customers can launch conversations in their preferred language. Learn more here.
  • End call tool: Let the agent automatically end the call with our new “End Call” tool. Learn more here
  • Flash default: Flash, our lowest latency model, is now the default for new agents. In your agent dashboard under “voice”, you can toggle between Turbo and Flash. Learn more about Flash here.
  • Privacy: Set concurrent call and daily call limits, turn off audio recordings, add feedback collection, and define customer terms & conditions.
  • Increased tool limits: Increase the number of tools available to your agent from 5 to 15. Learn more here.