`wss://api.elevenlabs.io/v1/text-to-speech/{voice_id}/stream-input?model_id={model}`
### Setting Up a Public URL for Your Server
To make your server accessible, create a public URL using a tunneling tool like ngrok:
```shell
ngrok http --url=
### Configuring Elevenlabs CustomLLM
Now let's make the changes in Elevenlabs
Direct your server URL to ngrok endpoint, also setup "Limit token usage" to 5000.
You can start interacting with Conversational AI with your own LLM server
# Knowledge Base
Learn how to enhance your conversational agent with custom knowledge
Knowledge bases allow you to provide additional context to your conversational
agent beyond its base LLM knowledge.
### 2. URL Import
## Best Practices
### Webhooks
Next, look for "Tools" in the "Agent" subnav. Add a new Tool to configure your webhook. For our AI receptionist, we created two Tools to interact with the Cal.com API:
### Path Parameters
You can add path parameters by including variables surrounded by curly brackets in your URL like this {variable}. Once added to the URL path, it will appear under Path Parameters with the ability to update the Data Type and Description.
Our AI receptionist does not call for Path Parameters so we will not be defining any.
### Query Parameters
Get and Delete requests typically have query parameters while Post and Patch do not. Our Get\_Available\_Slots tool relies on a Get request that requires the following query parameters: startTime, endTime, eventTypeId, eventTypeSlug, and duration.
In our Description for each, we define a prompt that our Conversational Agent will use to extract the relevant information from the call transcript using an LLM.
Here's how we defined our query parameters for our AI receptionist:
| Identifier | Data Type | Required | Description |
| ------------- | --------- | -------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| startTime | String | Yes | The start time of the slot the person is checking availability for in UTC timezone, formatted as ISO 8601 (e.g., '2024-08-13T09:00:00Z'). Extract time from natural language and convert to UTC. |
| endTime | String | Yes | The end time of the slot the person is checking availability for in UTC timezone, formatted as ISO 8601 (e.g., '2024-08-13T09:00:00Z'). Extract time from natural language and convert to UTC. |
| eventTypeSlug | String | Yes | The desired meeting length. Should be 15minutes, 30minutes, or 60minutes. |
| eventTypeId | Number | Yes | The desired meeting length, as an event id. If 15 minutes, return 1351800. If 30 minutes, return 1351801. If 60 minutes, return 1351802. |
### Body Parameters
Post and Patch requests typically have body parameters while Get and Delete do not. Our Book\_Meeting tool is a Post request and requires the following Body Parameters: startTime, eventTypeId, attendee.
In our Description for each, we define a prompt that our Conversational Agent will use to extract the relevant information from the call transcript using an LLM.
Here's how we defined our body parameters for our AI receptionist:
| Identifier | Data Type | Required | Description |
| ----------- | --------- | -------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| startTime | String | Yes | The start time of the slot the person is checking availability for in UTC timezone, formatted as ISO 8601 (e.g., '2024-08-13T09:00:00Z'). Extract time from natural language and convert to UTC. |
| eventTypeId | String | Yes | The unique Cal event ID for the meeting duration. Use 1351800 for a 15-minute meeting, 1351801 for 30 minutes, and 1351802 for 60 minutes. If no specific duration is provided, default to 1351801 (30 minutes). |
| attendee | Object | Yes | The info on the attendee including their full name, email address and time zone. |
Since attendee is an object, it's subfields are defined as their own parameters:
| Identifier | Data Type | Required | Description |
| ---------- | --------- | -------- | --------------------------------------------------------------------------------------------------------------- |
| name | String | Yes | The full name of the person booking the meeting. |
| email | String | Yes | The email address of the person booking the meeting. Should be a properly formatted email. |
| timeZone | String | Yes | The caller's timezone. Should be in the format of 'Continent/City' like 'Europe/London' or 'America/New\_York'. |
### Adjusting System Prompt to reference your Tools
Now that you've defined your Tools, instruct your agent on when and how to invoke them in your system prompt. If your Tools require the user to provide information, it's best to ask your agent to collect that information before calling it (though in many cases your agent will be able to realize it is missing information and will request for it anyway).
Here's the System Prompt we use for our AI Receptionist:
> You are my receptionist and people are calling to book a time with me.
>
> You can check my availability by using Get\_Available\_Slots. That endpoint takes start and end date/time and returns open slots in between. If someone asks for my availability but doesn't specify a date / time, just check for open slots tomorrow. If someone is checking availability and there are no open slots, keep checking the next day until you find one with availability.
>
> Once you've agreed upon a time to meet, you can use Book\_Meeting to book a call.
> You will need to collect their full name, the time they want to meet, whether they want to meet for 15, 30 or 60 minutes, and their email address to book a meeting.
>
> If you call Book\_Meeting and it fails, it's likely either that the email address is formatted in an invalid way or the selected time is not one where I am available.
# Agent Setup
Deploy customized, conversational voice agents in minutes.
## The Web Dashboard
The easiest way to get started with ElevenLabs Conversational AI is through our web dashboard.
You can access your dashboard [here](https://elevenlabs.io/sign-up). The web dashboard enables you to:
* Create and manage AI assistants
* Configure voice settings and conversation parameters
* Review conversation analytics and transcripts
* Manage API keys and integration settings
Status: {conversation.status}
Agent is {conversation.isSpeaking ? 'speaking' : 'listening'}
ElevenLabs Conversational AI
Status: Disconnected
Agent is listening
### Set Input Audio Format
Navigate to your agent -> Go to Advanced Section -> Select "μ-law 8000 Hz"
## Project Setup
2. Access the IAM (Identity and Access Management) Dashboard: You can find IAM under "Security, Identity, & Compliance" on the services menu. The IAM dashboard manages access to your AWS services securely.
3. Create a New User (if necessary): On the IAM dashboard, select "Users" and then "Add user". Enter a user name.
4. Set the permissions: attach policies directly to the user according to the access level you wish to grant. For S3 uploads, you can use the AmazonS3FullAccess policy. However, it's best practice to grant least privilege, or the minimal permissions necessary to perform a task. You might want to create a custom policy that specifically allows only the necessary actions on your S3 bucket.
5. Review and create the user: Review your settings and create the user. Upon creation, you'll be presented with an access key ID and a secret access key. Be sure to download and securely save these credentials; the secret access key cannot be retrieved again after this step.
6. Get AWS region name: ex. us-east-1
If you do not have an AWS S3 bucket, you will need to create a new one by following these steps:
1. Access the S3 dashboard: You can find S3 under "Storage" on the services menu.
2. Create a new bucket: On the S3 dashboard, click the "Create bucket" button.
3. Enter a bucket name and click on the "Create bucket" button. You can leave the other bucket options as default. The newly added bucket will appear in the list.
## Update your environment variables
Update the `.env` file with your ngrok domain and ElevenLabs API key.
```
# .env
SERVER_DOMAIN=*******.ngrok.app
ELEVENLABS_API_KEY=*************************
```
## Start the application
Run the following command to start the app:
```
npm run dev
```
## Set up Twilio
Follow Twilio’s guides to create a new number. Once you’ve created your number, navigate to the “Configure” tab in Phone Numbers -> Manage -> Active numbers
In the “A call comes in” section, enter the full URL to your application (make sure to add the`/call/incoming` path):
E.g. https\://\*\*\*\*\*\*\*ngrok.app/call/incoming
## Make a phone call
Make a call to your number. You should hear a message using the ElevenLabs voice.
## Tips for deploying to production
When running the application in production, make sure to set the `SERVER_DOMAIN` environment variable to that of your server. Be sure to also update the URL in Twilio to point to your production server.
## Conclusion
You should now have a basic understanding of integrating Twilio with ElevenLabs voices. If you have any further questions, or suggestions on how to improve this blog post, please feel free to select the “Suggest edits” or “Raise issue” button below.
# Models
ElevenLabs is the leading provider of AI-powered audio technology. This guide helps developers choose the right model for their use case.
• *Need real-time?* Use Turbo models
• *Multiple languages?* → Use `eleven_multilingual_v2` or `eleven_turbo_v2_5`
• *Conversational AI* → `eleven_turbo_v2_5`
• *Professional Voice Clones* → Either model{" "}
• *Speech to Speech?* → `eleven_english_sts_v2` or `eleven_multilingual_sts_v2` family
This is the code snippet that is used to embed Audio Native on a normal website such as Wordpress, Ghost, or Webflow. However, you can't use this snippet directly in React.
## Creating the Audio Native React component
Here's a handy component that you can reuse across your project:
{children ? children : 'Elevenlabs AudioNative Player'}
);
};
export default ElevenLabsAudioNative;
```
Here's a link to the component on GitHub - [ElevenLabsAudioNative.tsx](https://github.com/elevenlabs/elevenlabs-examples/blob/main/examples/audio-native/react/ElevenLabsAudioNative.tsx)
{children ? children : "Elevenlabs AudioNative Player"}
```
Here is our main div element which will be where our Audio Native player will be placed. The children of the component can be used to show content before the player has been loaded (e.g. Loading audio player…).
This public user ID is used to identify your Audio Native project.
## Use the Audio Native component
Now that you have the public user ID, you can use the component on your page. Simply import it, then pass it the public user ID from the previous step.
```tsx
import { ElevenLabsAudioNative } from "./path/to/ElevenLabsAudioNative";
export default function Page() {
return (
Your Blog Post Title
Your blog post...
### Troubleshooting
If you don't see the Audio Native player, try doing a hard refresh. This can sometimes be an issue because of the development server not properly reloading the script.
In Chrome it's: (⌘ or Ctrl) + Shift + R
### Why am I seeing “URL not allowed”?
Here's what's happening behind the scenes. Remember that script we loaded in the useEffect hook? This script is trying to scrape the content from your page to get all the text and convert it to audio. However, it can't load your page because it's on `localhost`. Audio Native can only process pages that are publicly accessible on the internet.
## Local testing with ngrok
This is where a service such as ngrok can help us. ngrok is a way to get your site on localhost to map to a public URL on the internet. They have a free tier, so visit their website [https://ngrok.com](https://ngrok.com), create an account and install it.
Here's their getting started guide - [https://ngrok.com/docs/getting-started](https://ngrok.com/docs/getting-started)
Once you have it installed, you can use a command similar to the one below to point your local React project to a public URL with ngrok. I'm running Next.js locally on port `3000`, so here's the command I run. Your details may vary.
```
ngrok http http://localhost:3000
```
Running this command will give you a URL that you can use in the next step.
### Update the allowed URLs to include the ngrok URL
Go to the Audio Native section:
[https://elevenlabs.io/audio-native](https://elevenlabs.io/audio-native)
Select the “My Websites” tab.
Enter the ngrok URL (from the previous step) in the “Allowed URLs” section.
This ensures that your player can only show on websites that you specify. This is very important, as someone else may otherwise be able to use your public user ID on their website.
Now visit your ngrok URL, you should see Audio Native processing your content. In the background, we are creating a project in your ElevenLabs account just for your page. This project contains the text from your page and converts it to audio.
View the newly created project here:
[https://elevenlabs.io/app/projects](https://elevenlabs.io/app/projects)
## Deploy to production
Make sure to also add the URL of your website to the allowed URLs once you've deployed your React app and you're ready to push to production.
We only used ngrok for local development, it's not needed for public facing URLs as ElevenLabs will directly grab the content from the website.
## Updating audio content
When updating the content on a page, you may notice that the audio from the Audio Native player won't update automatically.
In order to update the audio you'll have to go to the project in ElevenLabs and update the content from there manually. [https://elevenlabs.io/app/projects](https://elevenlabs.io/app/projects)
## Conclusion
Now that you have Audio Native working in your React project, go ahead and add the component to more pages on your website to begin converting content into high quality audio for your visitors.
# How to set up an ElevenLabs audio player for your articles in Framer
Before adding Audio Native to Framer, you'll need to create & customize your player, whitelist your blog's domain, and copy your embed code. If you need help completing those steps, refer to our [Audio Native overview](https://elevenlabs.io/docs/audio-native/overview).
Now that you've created & customized your Audio Native player, navigate to Framer. Go to Site Settings in Extract the \