Language detection | ElevenLabs Documentation

Overview

The language detection system tool allows your ElevenLabs agent to switch its output language to any the agent supports. This system tool is not enabled automatically. Its description can be customized to accommodate your specific use case.

Where possible, we recommend enabling all languages for an agent and enabling the language detection system tool.

Our language detection tool triggers language switching in two cases, both based on the received audio’s detected language and content:

detection if a user speaks a different language than the current output language, a switch will be triggered
content if the user asks in the current language to change to a new language, a switch will be triggered

Purpose: Automatically switch to the user’s detected language during conversations.

Trigger conditions: The LLM should call this tool when:

User speaks in a different language than the current conversation language
User explicitly requests to switch languages
Multi-language support is needed for the conversation

Parameters:

reason (string, required): The reason for the language switch
language (string, required): The language code to switch to (must be in supported languages list)

Function call format:

1 {
2   "type": "function",
3   "function": {
4     "name": "language_detection",
5     "arguments": "{\"reason\": \"User requested Spanish\", \"language\": \"es\"}"
6   }
7 }

Implementation: Configure supported languages in agent settings and add the language detection system tool. The agent will automatically switch voice and responses to match detected languages.

Enabling language detection

Configure supported languages

The languages that the agent can switch to must be defined in the Agent settings tab.

Add the language detection tool

Enable language detection by selecting the pre-configured system tool to your agent’s tools in the Agent tab. This is automatically available as an option when selecting add tool.

Configure tool description

Add a description that specifies when to call the tool

Restricting switching to the start of the conversation

In a long conversation, background speech or an unusual accent can occasionally trigger a switch the caller did not intend. Enabling only_at_conversation_start confines switching to the first two user turns.

If the language does not switch during those two turns, language detection is disabled for the rest of the conversation: every later attempt fails and the agent keeps speaking the current language. If the language does switch within that window, switching stays available for the rest of the conversation, so an incorrect detection can still be corrected.

The option is disabled by default.

Update via the dashboard

Update via the CLI

Update via the API

Select the Detect language tool in the Tools tab and enable Only at start of conversation.

When switching is blocked, the tool returns an error to the LLM instructing it to keep speaking the current language. The reason is not exposed to the caller.

API Implementation

When creating an agent via API, you can add the language detection tool to your agent configuration. It should be defined as a system tool:

1 from elevenlabs import (
2     ConversationalConfig,
3     ElevenLabs,
4     AgentConfig,
5     PromptAgent,
6     PromptAgentInputToolsItem_System,
7     LanguagePresetInput,
8     ConversationConfigClientOverrideInput,
9     AgentConfigOverride,
10 )
11 
12 # Initialize the client
13 elevenlabs = ElevenLabs(api_key="YOUR_API_KEY")
14 
15 # Create the language detection tool
16 language_detection_tool = PromptAgentInputToolsItem_System(
17     name="language_detection",
18     description=""  # Optional: Customize when the tool should be triggered
19 )
20 
21 # Create language presets
22 language_presets = {
23     "nl": LanguagePresetInput(
24         overrides=ConversationConfigClientOverrideInput(
25             agent=AgentConfigOverride(
26                 prompt=None,
27                 first_message="Hoi, hoe gaat het met je?",
28                 language=None
29             ),
30             tts=None
31         ),
32         first_message_translation=None
33     ),
34     "fi": LanguagePresetInput(
35         overrides=ConversationConfigClientOverrideInput(
36             agent=AgentConfigOverride(
37                 first_message="Hei, kuinka voit?",
38             ),
39             tts=None
40         ),
41     ),
42     "tr": LanguagePresetInput(
43         overrides=ConversationConfigClientOverrideInput(
44             agent=AgentConfigOverride(
45                 prompt=None,
46                 first_message="Merhaba, nasılsın?",
47                 language=None
48             ),
49             tts=None
50         ),
51     ),
52     "ru": LanguagePresetInput(
53         overrides=ConversationConfigClientOverrideInput(
54             agent=AgentConfigOverride(
55                 prompt=None,
56                 first_message="Привет, как ты?",
57                 language=None
58             ),
59             tts=None
60         ),
61     ),
62     "pt": LanguagePresetInput(
63         overrides=ConversationConfigClientOverrideInput(
64             agent=AgentConfigOverride(
65                 prompt=None,
66                 first_message="Oi, como você está?",
67                 language=None
68             ),
69             tts=None
70         ),
71     )
72 }
73 
74 # Create the agent configuration
75 conversation_config = ConversationalConfig(
76     agent=AgentConfig(
77         prompt=PromptAgent(
78             tools=[language_detection_tool],
79             first_message="Hi how are you?"
80         )
81     ),
82     language_presets=language_presets
83 )
84 
85 # Create the agent
86 response = elevenlabs.conversational_ai.agents.create(
87     conversation_config=conversation_config
88 )

Leave the description blank to use the default language detection prompt.