Vite (Javascript)

Learn how to create a web application that enables voice conversations with ElevenLabs AI agents

This tutorial will guide you through creating a web client that can interact with a Conversational AI agent. You’ll learn how to implement real-time voice conversations, allowing users to speak with an AI agent that can listen, understand, and respond naturally using voice synthesis.

Looking to build with React/Next.js? Check out our Next.js guide

What You’ll Need

  1. An ElevenLabs agent created following this guide
  2. npm installed on your local system
  3. Basic knowledge of JavaScript

Looking for a complete example? Check out our Vanilla JS demo on GitHub.

Project Setup

1

Create a Project Directory

Open a terminal and create a new directory for your project:

$mkdir elevenlabs-conversational-ai
>cd elevenlabs-conversational-ai
2

Initialize npm and Install Dependencies

Initialize a new npm project and install the required packages:

$npm init -y
>npm install vite @11labs/client
3

Set up Basic Project Structure

Add this to your package.json:

1{
2    "scripts": {
3        ...
4        "dev:frontend": "vite"
5    }
6}

Create the following file structure:

$elevenlabs-conversational-ai/
>├── index.html
>├── script.js
>├── package-lock.json
>├── package.json
>└── node_modules

Implementing the Voice Chat Interface

1

Create the HTML Interface

In index.html, set up a simple user interface:

index.html
1<!DOCTYPE html>
2<html lang="en">
3    <head>
4        <meta charset="UTF-8" />
5        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
6        <title>ElevenLabs Conversational AI</title>
7    </head>
8    <body style="font-family: Arial, sans-serif; text-align: center; padding: 50px;">
9        <h1>ElevenLabs Conversational AI</h1>
10        <div style="margin-bottom: 20px;">
11            <button id="startButton" style="padding: 10px 20px; margin: 5px;">Start Conversation</button>
12            <button id="stopButton" style="padding: 10px 20px; margin: 5px;" disabled>Stop Conversation</button>
13        </div>
14        <div style="font-size: 18px;">
15            <p>Status: <span id="connectionStatus">Disconnected</span></p>
16            <p>Agent is <span id="agentStatus">listening</span></p>
17        </div>
18        <script type="module" src="../images/script.js"></script>
19    </body>
20</html>
2

Implement the Conversation Logic

In script.js, implement the functionality:

script.js
1import { Conversation } from '@11labs/client';
2
3const startButton = document.getElementById('startButton');
4const stopButton = document.getElementById('stopButton');
5const connectionStatus = document.getElementById('connectionStatus');
6const agentStatus = document.getElementById('agentStatus');
7
8let conversation;
9
10async function startConversation() {
11    try {
12        // Request microphone permission
13        await navigator.mediaDevices.getUserMedia({ audio: true });
14
15        // Start the conversation
16        conversation = await Conversation.startSession({
17            agentId: 'YOUR_AGENT_ID', // Replace with your agent ID
18            onConnect: () => {
19                connectionStatus.textContent = 'Connected';
20                startButton.disabled = true;
21                stopButton.disabled = false;
22            },
23            onDisconnect: () => {
24                connectionStatus.textContent = 'Disconnected';
25                startButton.disabled = false;
26                stopButton.disabled = true;
27            },
28            onError: (error) => {
29                console.error('Error:', error);
30            },
31            onModeChange: (mode) => {
32                agentStatus.textContent = mode.mode === 'speaking' ? 'speaking' : 'listening';
33            },
34        });
35    } catch (error) {
36        console.error('Failed to start conversation:', error);
37    }
38}
39
40async function stopConversation() {
41    if (conversation) {
42        await conversation.endSession();
43        conversation = null;
44    }
45}
46
47startButton.addEventListener('click', startConversation);
48stopButton.addEventListener('click', stopConversation);
3

Start the frontend server

$npm run dev:frontend
Make sure to replace 'YOUR_AGENT_ID' with your actual agent ID from ElevenLabs.

This authentication step is only required for private agents. If you’re using a public agent, you can skip this section and directly use the agentId in the startSession call.

1

Create Environment Variables

Create a .env file in your project root:

.env
1ELEVENLABS_API_KEY=your-api-key-here
2AGENT_ID=your-agent-id-here

Make sure to add .env to your .gitignore file to prevent accidentally committing sensitive credentials.

2

Setup the Backend

  1. Install additional dependencies:
$npm install express cors dotenv
  1. Create a new folder called backend:
$elevenlabs-conversational-ai/
>├── backend
>...
3

Create the Server

backend/server.js
1require("dotenv").config();
2
3const express = require("express");
4const cors = require("cors");
5
6const app = express();
7app.use(cors());
8app.use(express.json());
9
10const PORT = process.env.PORT || 3001;
11
12app.get("/api/get-signed-url", async (req, res) => {
13    try {
14        const response = await fetch(
15            `https://api.elevenlabs.io/v1/convai/conversation/get_signed_url?agent_id=${process.env.AGENT_ID}`,
16            {
17                headers: {
18                    "xi-api-key": process.env.ELEVENLABS_API_KEY,
19                },
20            }
21        );
22
23        if (!response.ok) {
24            throw new Error("Failed to get signed URL");
25        }
26
27        const data = await response.json();
28        res.json({ signedUrl: data.signed_url });
29    } catch (error) {
30        console.error("Error:", error);
31        res.status(500).json({ error: "Failed to generate signed URL" });
32    }
33});
34
35app.listen(PORT, () => {
36    console.log(`Server running on http://localhost:${PORT}`);
37});
4

Update the Client Code

Modify your script.js to fetch and use the signed URL:

1// ... existing imports and variables ...
2
3async function getSignedUrl() {
4    const response = await fetch('http://localhost:3001/api/get-signed-url');
5    if (!response.ok) {
6        throw new Error(`Failed to get signed url: ${response.statusText}`);
7    }
8    const { signedUrl } = await response.json();
9    return signedUrl;
10}
11
12async function startConversation() {
13    try {
14        await navigator.mediaDevices.getUserMedia({ audio: true });
15
16        const signedUrl = await getSignedUrl();
17
18        conversation = await Conversation.startSession({
19            signedUrl,
20            // agentId has been removed...
21            onConnect: () => {
22                connectionStatus.textContent = 'Connected';
23                startButton.disabled = true;
24                stopButton.disabled = false;
25            },
26            onDisconnect: () => {
27                connectionStatus.textContent = 'Disconnected';
28                startButton.disabled = false;
29                stopButton.disabled = true;
30            },
31            onError: (error) => {
32                console.error('Error:', error);
33            },
34            onModeChange: (mode) => {
35                agentStatus.textContent = mode.mode === 'speaking' ? 'speaking' : 'listening';
36            },
37        });
38    } catch (error) {
39        console.error('Failed to start conversation:', error);
40    }
41}
42
43// ... rest of the code ...

Signed URLs expire after a short period. However, any conversations initiated before expiration will continue uninterrupted. In a production environment, implement proper error handling and URL refresh logic for starting new conversations.

5

Update the package.json

1{
2    "scripts": {
3        ...
4        "dev:backend": "node backend/server.js",
5        "dev": "npm run dev:frontend & npm run dev:backend"
6    }
7}
6

Run the Application

Start the application with:

$npm run dev

Next Steps

Now that you have a basic implementation, you can:

  1. Add visual feedback for voice activity
  2. Implement error handling and retry logic
  3. Add a chat history display
  4. Customize the UI to match your brand

For more advanced features and customization options, check out the @11labs/client package.

Built with