iOS SDK

Conversational AI SDK: deploy customized, interactive voice agents in your Swift applications.

Also see the Conversational AI overview

Installation

Add the ElevenLabs Swift SDK to your project using Swift Package Manager:

1

Add the Package Dependency

  1. Open your project in Xcode
  2. Go to File > Add Packages...
  3. Enter the repository URL: https://github.com/elevenlabs/ElevenLabsSwift
  4. Select your desired version
2

Import the SDK

1import ElevenLabsSDK

Ensure you add NSMicrophoneUsageDescription to your Info.plist to explain microphone access to users.

Usage

This library is primarily designed for Conversational AI integration in Swift applications. Please use an alternative dependency for other features, such as speech synthesis.

Initialize Conversation

First, create a session configuration and set up the necessary callbacks:

1// Configure the session
2let config = ElevenLabsSDK.SessionConfig(agentId: "your-agent-id")
3
4// Set up callbacks
5var callbacks = ElevenLabsSDK.Callbacks()
6callbacks.onConnect = { conversationId in
7    print("Connected with ID: \(conversationId)")
8}
9callbacks.onDisconnect = {
10    print("Disconnected")
11}
12callbacks.onMessage = { message, role in
13    print("\(role.rawValue): \(message)")
14}
15callbacks.onError = { error, info in
16    print("Error: \(error), Info: \(String(describing: info))")
17}
18callbacks.onStatusChange = { status in
19    print("Status changed to: \(status.rawValue)")
20}
21callbacks.onModeChange = { mode in
22    print("Mode changed to: \(mode.rawValue)")
23}
24callbacks.onVolumeUpdate = { volume in
25    print("Volume updated: \(volume)")
26}

Session Configuration

There are two ways to initialize a session:

You can obtain an Agent ID through the ElevenLabs UI:

1let config = ElevenLabsSDK.SessionConfig(agentId: "<your-agent-id>")

Client Tools

Client Tools allow you to register custom functions that can be called by your AI agent during conversations. This enables your agent to perform actions in your application.

Registering Tools

Register custom tools before starting a conversation:

1// Create client tools instance
2var clientTools = ElevenLabsSDK.ClientTools()
3
4// Register a custom tool with an async handler
5clientTools.register("generate_joke") { parameters async throws -> String? in
6    // Parameters is a [String: Any] dictionary
7    guard let joke = parameters["joke"] as? String else {
8        throw ElevenLabsSDK.ClientToolError.invalidParameters
9    }
10    print("generate_joke tool received joke: \(joke)")
11
12    return joke
13}

Remember to setup your agent with the client-tools in the ElevenLabs UI. See the Client Tools documentation for setup instructions.

Starting the Conversation

Initialize the conversation session asynchronously:

1Task {
2    do {
3        let conversation = try await ElevenLabsSDK.Conversation.startSession(
4            config: config,
5            callbacks: callbacks,
6            clientTools: clientTools // Optional: pass the previously configured client tools
7        )
8        // Use the conversation instance
9    } catch {
10        print("Failed to start conversation: \(error)")
11    }
12}

The client tools parameter is optional. If you don’t need custom tools, you can omit it when starting the session.

Audio Sample Rates

The ElevenLabs SDK currently uses a default input sample rate of 16,000 Hz. However, the output sample rate is configurable based on the agent’s settings. Ensure that the output sample rate aligns with your specific application’s audio requirements for smooth interaction.

The SDK does not currently support ulaw format for audio encoding. For compatibility, consider using alternative formats.

Managing the Session

1// Starts the session
2conversation.startSession()
3// Ends the session
4conversation.endSession()

Example Implementation

For a full, working example, check out the example application on GitHub.

Here’s an example SwiftUI view implementing the conversation interface:

1struct ConversationalAIView: View {
2    @State private var conversation: ElevenLabsSDK.Conversation?
3    @State private var mode: ElevenLabsSDK.Mode = .listening
4    @State private var status: ElevenLabsSDK.Status = .disconnected
5    @State private var audioLevel: Float = 0.0
6
7    private func startConversation() {
8        Task {
9            do {
10                let config = ElevenLabsSDK.SessionConfig(agentId: "your-agent-id")
11                var callbacks = ElevenLabsSDK.Callbacks()
12
13                callbacks.onConnect = { conversationId in
14                    status = .connected
15                }
16                callbacks.onDisconnect = {
17                    status = .disconnected
18                }
19                callbacks.onModeChange = { newMode in
20                    DispatchQueue.main.async {
21                        mode = newMode
22                    }
23                }
24                callbacks.onVolumeUpdate = { newVolume in
25                    DispatchQueue.main.async {
26                        audioLevel = newVolume
27                    }
28                }
29
30                conversation = try await ElevenLabsSDK.Conversation.startSession(
31                    config: config,
32                    callbacks: callbacks
33                )
34            } catch {
35                print("Failed to start conversation: \(error)")
36            }
37        }
38    }
39
40    var body: some View {
41        VStack {
42            // Your UI implementation
43            Button(action: startConversation) {
44                Text(status == .connected ? "End Call" : "Start Call")
45            }
46        }
47    }
48}

This SDK is currently experimental and under active development. While it’s stable enough for testing and development, it’s not recommended for production use yet.