Unity SDK

The ElevenAgents Unity SDK connects a Unity scene to an ElevenLabs agent. It captures the player’s microphone, streams the agent’s voice back through an AudioSource, and exposes the session through a single Conversation API that behaves the same in the Editor, desktop standalone players, and WebGL builds.

Requirements

• Unity 6.3 LTS (6000.3.0f1) or later. The SDK uses scriptable-audio APIs introduced in Unity 6.3.
• ApiCompatibilityLevel set to .NET Standard 2.1 (ApiCompatibilityLevel.NET_Standard), the default for Unity 2023.1 and later. In the Editor, find it under Edit > Project Settings > Player > Other Settings > Api Compatibility Level.
• An ElevenLabs account and an agent. Create one in the dashboard or follow the quickstart.

Platform support

The SDK selects the transport for the active build target automatically. WebRTC is only available on WebGL; on every other target ConnectionType.WebRTC throws a NotSupportedException.

Beta targets use the same native transport as desktop but are not yet covered by the SDK’s test suite. Report issues on GitHub.

Installation

In Unity, open Window > Package Manager, select + > Install package from git URL, and paste the repository URL:

https://github.com/elevenlabs/unity.git

The Package Manager installs the package as ElevenAgents (io.elevenlabs.agents), together with its dependencies: Input System and Newtonsoft Json.

Quickstart

This section adds a voice agent to an empty scene. The agent starts talking when you enter Play Mode and the session ends when you leave it.

using System;
using ElevenLabs.Agents;
using ElevenLabs.Protocol;
using UnityEngine;
public sealed class AgentConversation : MonoBehaviour
{
    [SerializeField]
    private string agentId = "agent_7101k5zvyjhmfg983brhmhkd98n6";
    private Conversation conversation;
    private async void Start()
    {
        try
        {
            conversation = await Conversation.StartSessionAsync(
                new ConversationOptions { AgentId = agentId }
            );
        }
        catch (Exception ex)
        {
            Debug.LogError($"Failed to start session: {ex.Message}");
            return;
        }
        Debug.Log($"Connected: {conversation.ConversationId}");
        conversation.UserTranscriptReceived += OnUserTranscript;
        conversation.AgentResponded += OnAgentResponse;
        conversation.ErrorOccurred += message => Debug.LogError(message);
        conversation.Disconnected += details => Debug.Log($"Disconnected: {details.Reason}");
    }
    private void OnUserTranscript(UserTranscriptArgs args) =>
        Debug.Log($"User: {args.UserTranscript}");
    private void OnAgentResponse(AgentResponseArgs args) =>
        Debug.Log($"Agent: {args.AgentResponse}");
    private void OnDestroy()
    {
        _ = conversation?.EndSession();
    }
}

Public and private agents

var conversation = await Conversation.StartSessionAsync(
    new ConversationOptions { AgentId = "agent_7101k5zvyjhmfg983brhmhkd98n6" }
);

Set exactly one of AgentId, SignedUrl, or ConversationToken. Set UserId to associate the conversation with a player in your own systems.

Spatial audio

By default the SDK plays the agent’s voice through a hidden AudioSource it creates. To place the voice in the scene, pass your own AudioSource as OutputAudioSource:

[SerializeField]
private AudioSource voiceSource; // Spatial Blend = 1 for 3D audio
var conversation = await Conversation.StartSessionAsync(
    new ConversationOptions
    {
        AgentId = "agent_7101k5zvyjhmfg983brhmhkd98n6",
        OutputAudioSource = voiceSource,
    }
);

The SDK keeps the source’s spatial blend, rolloff, mixer routing, and position. It restores the source’s clip, loop, and volume settings when the session ends.

To drive a visual effect from the agent’s voice, such as a mouth animation, read conversation.GetOutputVolume() each frame. It returns a value between 0 and 1 on every platform.

Personalization

Use dynamic variables to fill placeholders in the agent’s prompt and first message, and overrides to replace parts of the agent configuration for one session:

using System.Collections.Generic;
using ElevenLabs.Protocol;
var options = new ConversationOptions
{
    AgentId = "agent_7101k5zvyjhmfg983brhmhkd98n6",
    DynamicVariables = new Dictionary<string, object>
    {
        ["player_name"] = "Ada",
        ["quest_stage"] = 3,
    },
    Overrides = new ConversationConfigOverride
    {
        Agent = new ConversationConfigOverrideAgent
        {
            FirstMessage = "Welcome back, {{player_name}}.",
        },
        Tts = new ConversationConfigOverrideTts { VoiceId = "JBFqnCBsd6RMkjVDRZzb" },
    },
};

Dynamic variable values must be strings, numbers, or booleans. Each override must be enabled in the agent’s Security settings before the server accepts it.

Client tools

Client tools let the agent trigger actions in your game. Define the tool in the agent’s settings, then register a handler with the same name:

using Newtonsoft.Json;
public sealed class OpenDoorParams
{
    [JsonProperty("door_id")]
    public string DoorId { get; set; }
}
conversation.RegisterTool<OpenDoorParams, string>("open_door", parameters =>
{
    DoorRegistry.Open(parameters.DoorId);
    return $"Door {parameters.DoorId} is open.";
});

The SDK deserializes the tool parameters into TParams with Newtonsoft Json, so use [JsonProperty] when a parameter name differs from the property name. The return value is serialized as the tool result. Return an Awaitable<TResult> for asynchronous handlers. Throw ClientToolException with an errorType to report a specific failure to the agent; any other exception is reported as a generic error.

Calls to tools without a registered handler raise ErrorOccurred, or UnhandledClientToolCall if you subscribe to it.

Events

Subscribe to events on the Conversation returned by StartSessionAsync. The SDK raises them on Unity’s main thread.

The Connected event fires before StartSessionAsync returns, so read ConversationId and Status directly once the call completes. The package changelog lists the full event surface.

Methods

For example, send the player’s current location to the agent when they enter a new area:

conversation.SendContextualUpdate("The player entered the abandoned lighthouse.");

WebGL builds

WebGL builds need one additional Player Setting.

Unity has no scriptable audio pipeline on WebGL, so the SDK plays the agent’s voice through the browser’s Web Audio API and mirrors the volume, position, and spatial settings of your OutputAudioSource. As a result, AudioSource.GetOutputData and OnAudioFilterRead return silence on WebGL. Use GetOutputVolume() and GetOutputByteFrequencyData() instead.

See COMPATIBILITY.md for the full WebGL compatibility matrix.

Troubleshooting

• StartSessionAsync throws ArgumentException. Set exactly one of AgentId, SignedUrl, or ConversationToken.
• The session fails to connect for a valid agent ID. The agent requires authentication. Turn authentication off or connect with a signed URL.
• An override has no effect. Enable the matching override in the agent’s Security settings.
• The player hears nothing on WebGL. Browsers keep audio suspended until the player interacts with the page. Start the session after a click or key press.
• The microphone is unavailable on WebGL. Browsers only grant microphone access to pages served over HTTPS or from localhost.

Report bugs and request features on GitHub.