Client tools

Client tools enable your assistant to execute client-side functions. Unlike server-side tools, client tools allow the assistant to perform actions such as triggering browser events, running client-side functions, or sending notifications to a UI.

Overview

Applications may require assistants to interact directly with the user’s environment. Client-side tools give your assistant the ability to perform client-side operations.

Here are a few examples where client tools can be useful:

• Triggering UI events: Allow an assistant to trigger browser events, such as alerts, modals or notifications.
• Interacting with the DOM: Enable an assistant to manipulate the Document Object Model (DOM) for dynamic content updates or to guide users through complex interfaces.

Guide

Prerequisites

• An ElevenLabs account
• A configured ElevenLabs Conversational Agent (create one here)

1
from elevenlabs import ElevenLabs
2
from elevenlabs.conversational_ai.conversation import Conversation, ClientTools
3
4
def log_message(parameters):
5
    message = parameters.get("message")
6
    print(message)
7
8
client_tools = ClientTools()
9
client_tools.register("logMessage", log_message)
10
11
conversation = Conversation(
12
    client=ElevenLabs(api_key="your-api-key"),
13
    agent_id="your-agent-id",
14
    client_tools=client_tools,
15
    # ...
16
)
17
18
conversation.start_session()

Passing client tool results to the conversation context

When you want your agent to receive data back from a client tool, ensure that you tick the Wait for response option in the tool configuration.

Once the client tool is added, when the function is called the agent will wait for its response and append the response to the conversation context.

1
def get_customer_details():
2
    # Fetch customer details (e.g., from an API or database)
3
    customer_data = {
4
        "id": 123,
5
        "name": "Alice",
6
        "subscription": "Pro"
7
    }
8
    # Return the customer data; it can also be a JSON string if needed.
9
    return customer_data
10
11
client_tools = ClientTools()
12
client_tools.register("getCustomerDetails", get_customer_details)
13
14
conversation = Conversation(
15
    client=ElevenLabs(api_key="your-api-key"),
16
    agent_id="your-agent-id",
17
    client_tools=client_tools,
18
    # ...
19
)
20
21
conversation.start_session()

In this example, when the agent calls getCustomerDetails, the function will execute on the client and the agent will receive the returned data, which is then used as part of the conversation context.

Troubleshooting

Best practices

Name tools intuitively, with detailed descriptions

If you find the assistant does not make calls to the correct tools, you may need to update your tool names and descriptions so the assistant more clearly understands when it should select each tool. Avoid using abbreviations or acronyms to shorten tool and argument names.

You can also include detailed descriptions for when a tool should be called. For complex tools, you should include descriptions for each of the arguments to help the assistant know what it needs to ask the user to collect that argument.

Name tool parameters intuitively, with detailed descriptions

Use clear and descriptive names for tool parameters. If applicable, specify the expected format for a parameter in the description (e.g., YYYY-mm-dd or dd/mm/yy for a date).

Consider providing additional information about how and when to call tools in your assistant’s system prompt

Providing clear instructions in your system prompt can significantly improve the assistant’s tool calling accuracy. For example, guide the assistant with instructions like the following:

Use `check_order_status` when the user inquires about the status of their order, such as 'Where is my order?' or 'Has my order shipped yet?'.

Provide context for complex scenarios. For example:

Before scheduling a meeting with `schedule_meeting`, check the user's calendar for availability using check_availability to avoid conflicts.

LLM selection

It’s important to note that the choice of LLM matters to the success of function calls. Some LLMs can struggle with extracting the relevant parameters from the conversation.