Skip to navigation

Client tools

Empower your assistant to trigger client-side operations.

Client tools enable your assistant to execute client-side functions. Unlike webhook tools, client tools allow the assistant to perform actions such as triggering browser events, running client-side functions, or sending notifications to a UI.

Overview

Applications may require assistants to interact directly with the user’s environment. Client-side tools give your assistant the ability to perform client-side operations.

Here are a few examples where client tools can be useful:

  • Triggering UI events: Allow an assistant to trigger browser events, such as alerts, modals or notifications.
  • Interacting with the DOM: Enable an assistant to manipulate the Document Object Model (DOM) for dynamic content updates or to guide users through complex interfaces.

To call server-side APIs, or for anything whose result must be trusted, use webhook tools instead. See Trust and security.

Trust and security

“Client” means the application that holds the conversation’s connection. That’s usually a browser or mobile app, but it can also be a server process, for example one using Conversation in the Python SDK. When the agent calls a client tool, the call goes to whichever application is connected. The agent then uses the result that application sends back as if it were true.

So whether you can trust a client tool depends on where the conversation runs:

Where the conversation runsWho controls itWhat you can trust
A browser, mobile app, kiosk or other user deviceThe userNothing it sends back. The user can see every tool call and change any result.
Your own serverYouYour code, but not its input. Parameters come from the LLM, so what the caller says can steer them.

A useful test: if it would be a problem for someone to change the result in their browser’s developer tools, the tool shouldn’t be a client tool. Use a webhook tool instead.

Client tools on a user’s device

  • Don’t use the result to check anything. Payment status, identity, age, account balance or permissions reported by a client tool are whatever the user’s device says they are. Check these in a webhook tool, which ElevenLabs calls on your server and the user’s device never sees.
  • Don’t send the device anything the user shouldn’t see. Every parameter of a client tool call is sent to the device, including values filled from dynamic variables.
  • Don’t trust dynamic variables the device set. The user controls the values your page passes in when it starts the conversation, including through talk-to page URL parameters. If a webhook tool takes a customer_id from one of those, a user can ask for someone else’s data. Have your webhook work out who the user is itself. For example, pass a session token your backend issued in a secret dynamic variable, send it as a header, and check it on your server.
  • Never put an API key in client code. Connect with a signed URL or conversation token that your backend generates.

Client tools on your server

  • Treat parameters as untrusted input. Validate them and check that the caller is allowed to act on the record they name, the same way you would for a public API endpoint.
  • Keep state per conversation. If one process runs many conversations, give each conversation its own ClientTools, and don’t keep anything about a caller in global variables.
  • Don’t leak error messages. In the Python SDK, if a handler raises an exception, its message is sent to the agent as the tool’s result, and the agent may read it out to the caller. Catch exceptions in the handler and return a message that’s safe to share.

Choose a client tool or a webhook tool

Use a client tool toUse a webhook tool to
Change what the user sees: open a page, show a product, highlight part of a formRead or change data in your systems
Read state that only exists on the device, such as the current page or what’s on screenCheck anything that must be true, such as a payment, an identity or an account’s status
Run code you’d be happy for the user to run themselvesUse credentials such as API keys, OAuth or database access

Many agents use both. For example, a webhook tool places an order, and a client tool then shows the confirmation on screen.

Guide

Prerequisites

1

Create a new client-side tool

Configure a client tool named logMessage with a required string parameter message (“The message to log in the console”).

Navigate to your agent dashboard. In the Tools section, click Add Tool. Ensure the Tool Type is set to Client. Then configure the following:

SettingParameter
NamelogMessage
DescriptionUse this client-side tool to log a message to the user’s client.

Then create a new parameter message with the following configuration:

SettingParameter
Data TypeString
Identifiermessage
Requiredtrue
DescriptionThe message to log in the console. Ensure the message is informative and relevant.

logMessage client-tool setup

2

Register the client tool in your code

Unlike webhook tools, client tools need to be registered in your code.

Use the following code to register the client tool:

from elevenlabs import ElevenLabs
from elevenlabs.conversational_ai.conversation import Conversation, ClientTools
def log_message(parameters):
message = parameters.get("message")
print(message)
client_tools = ClientTools()
client_tools.register("logMessage", log_message)
conversation = Conversation(
client=ElevenLabs(api_key="your-api-key"),
agent_id="agent_7101k5zvyjhmfg983brhmhkd98n6",
requires_auth=True,
client_tools=client_tools,
# ...
)
conversation.start_session()

The tool and parameter names in the agent configuration are case-sensitive and must match those registered in your code.

3

Testing

Initiate a conversation with your agent and say something like:

Log a message to the console that says Hello World

You should see a Hello World log appear in your console.

4

Next steps

Now that you’ve set up a basic client-side event, you can:

  • Explore more complex client tools like opening modals, navigating to pages, or interacting with the DOM.
  • Combine client tools with server-side webhooks for full-stack interactions.
  • Use client tools to enhance user engagement and provide real-time feedback during conversations.

Passing client tool results to the conversation context

When you want your agent to receive data back from a client tool, ensure that you tick the Wait for response option in the tool configuration.

Wait for response option in client tool configuration

Once the client tool is added, when the function is called the agent will wait for its response and append the response to the conversation context.

def get_customer_details():
# Fetch customer details (e.g., from an API or database)
customer_data = {
"id": 123,
"name": "Alice",
"subscription": "Pro"
}
# Return the customer data; it can also be a JSON string if needed.
return customer_data
client_tools = ClientTools()
client_tools.register("getCustomerDetails", get_customer_details)
conversation = Conversation(
client=ElevenLabs(api_key="your-api-key"),
agent_id="agent_7101k5zvyjhmfg983brhmhkd98n6",
requires_auth=True,
client_tools=client_tools,
# ...
)
conversation.start_session()

In this example, when the agent calls getCustomerDetails, the function will execute on the client and the agent will receive the returned data, which is then used as part of the conversation context. The values from the response can also optionally be assigned to dynamic variables, similar to webhook tools. Note system tools cannot update dynamic variables.

Troubleshooting

  • Ensure the tool and parameter names in the agent configuration match those registered in your code.
  • View the conversation transcript in the agent dashboard to verify the tool is being executed.
  • Open the browser console to check for any errors.
  • Ensure that your code has necessary error handling for undefined or unexpected parameters.

Best practices

Name tools intuitively, with detailed descriptions

If you find the assistant does not make calls to the correct tools, you may need to update your tool names and descriptions so the assistant more clearly understands when it should select each tool. Avoid using abbreviations or acronyms to shorten tool and argument names.

You can also include detailed descriptions for when a tool should be called. For complex tools, you should include descriptions for each of the arguments to help the assistant know what it needs to ask the user to collect that argument.

Name tool parameters intuitively, with detailed descriptions

Use clear and descriptive names for tool parameters. If applicable, specify the expected format for a parameter in the description (e.g., YYYY-mm-dd or dd/mm/yy for a date).

Consider providing additional information about how and when to call tools in your assistant’s system prompt

Providing clear instructions in your system prompt can significantly improve the assistant’s tool calling accuracy. For example, guide the assistant with instructions like the following:

Use `check_order_status` when the user inquires about the status of their order, such as 'Where is my order?' or 'Has my order shipped yet?'.

Provide context for complex scenarios. For example:

Before scheduling a meeting with `schedule_meeting`, check the user's calendar for availability using check_availability to avoid conflicts.

LLM selection

When using tools, we recommend picking high intelligence models like GPT 6 or Claude Sonnet 5.5.

It’s important to note that the choice of LLM matters to the success of function calls. Some LLMs can struggle with extracting the relevant parameters from the conversation.