Currently, the following models are natively supported and can be configured via the agent settings:

Gemini 2.0 Flash
Gemini 1.5 Flash
Gemini 1.5 Pro
Gemini 1.0 Pro
GPT-4o Mini
GPT-4o
GPT-4 Turbo
GPT-3.5 Turbo
Claude 3.5 Sonnet
Claude 3 Haiku

Supported models

Custom LLMs let you bring your own OpenAI API key or run an entirely custom LLM server.

Overview

By default, we use our own internal credentials for popular models like OpenAI. To use a custom LLM server, it must align with the OpenAI create chat completion request/response structure.

The following guides cover both use cases:

Bring your own OpenAI key: Use your own OpenAI API key with our platform.
Custom LLM server: Host and connect your own LLM server implementation.

You’ll learn how to:

Store your OpenAI API key in ElevenLabs
host a server that replicates OpenAI’s create chat completion endpoint
Direct ElevenLabs to your custom endpoint
Pass extra parameters to your LLM as needed

Using your own OpenAI key

To integrate a custom OpenAI key, create a secret containing your OPENAI_API_KEY:

Navigate to the “Secrets” page and select “Add Secret”

Choose “Custom LLM” from the dropdown menu.

Enter the URL, your model, and the secret you created.

Set “Custom LLM extra body” to true.

Custom LLM Server

To bring a custom LLM server, set up a compatible server endpoint using OpenAI’s style, specifically targeting create_chat_completion.

Here’s an example server implementation using FastAPI and OpenAI’s Python SDK:

1 import json
2 import os
3 import fastapi
4 from fastapi.responses import StreamingResponse
5 from openai import AsyncOpenAI
6 import uvicorn
7 import logging
8 from dotenv import load_dotenv
9 from pydantic import BaseModel
10 from typing import List, Optional
11 
12 # Load environment variables from .env file
13 load_dotenv()
14 
15 # Retrieve API key from environment
16 OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')
17 if not OPENAI_API_KEY:
18     raise ValueError("OPENAI_API_KEY not found in environment variables")
19 
20 app = fastapi.FastAPI()
21 oai_client = AsyncOpenAI(api_key=OPENAI_API_KEY)
22 
23 class Message(BaseModel):
24     role: str
25     content: str
26 
27 class ChatCompletionRequest(BaseModel):
28     messages: List[Message]
29     model: str
30     temperature: Optional[float] = 0.7
31     max_tokens: Optional[int] = None
32     stream: Optional[bool] = False
33     user_id: Optional[str] = None
34 
35 @app.post("/v1/chat/completions")
36 async def create_chat_completion(request: ChatCompletionRequest) -> StreamingResponse:
37     oai_request = request.dict(exclude_none=True)
38     if "user_id" in oai_request:
39         oai_request["user"] = oai_request.pop("user_id")
40 
41     chat_completion_coroutine = await oai_client.chat.completions.create(**oai_request)
42 
43     async def event_stream():
44         try:
45             async for chunk in chat_completion_coroutine:
46                 # Convert the ChatCompletionChunk to a dictionary before JSON serialization
47                 chunk_dict = chunk.model_dump()
48                 yield f"data: {json.dumps(chunk_dict)}\n\n"
49             yield "data: [DONE]\n\n"
50         except Exception as e:
51             logging.error("An error occurred: %s", str(e))
52             yield f"data: {json.dumps({'error': 'Internal error occurred!'})}\n\n"
53 
54     return StreamingResponse(event_stream(), media_type="text/event-stream")
55 
56 if __name__ == "__main__":
57     uvicorn.run(app, host="0.0.0.0", port=8013)

Run this code or your own server code.

Setting Up a Public URL for Your Server

To make your server accessible, create a public URL using a tunneling tool like ngrok:

$ ngrok http --url=<Your url>.ngrok.app 8013

Configuring Elevenlabs CustomLLM

Now let’s make the changes in Elevenlabs

Direct your server URL to ngrok endpoint, setup “Limit token usage” to 5000 and set “Custom LLM extra body” to true.

You can start interacting with Conversational AI with your own LLM server

Additional Features

Custom LLM Parameters

You may pass additional parameters to your custom LLM implementation.

Python

Define the Extra Parameters

Create an object containing your custom parameters:

1 from elevenlabs.conversational_ai.conversation import Conversation, ConversationConfig
2 
3 extra_body_for_convai = {
4     "UUID": "123e4567-e89b-12d3-a456-426614174000",
5     "parameter-1": "value-1",
6     "parameter-2": "value-2",
7 }
8 
9 config = ConversationConfig(
10     extra_body=extra_body_for_convai,
11 )

Update the LLM Implementation

Modify your custom LLM code to handle the additional parameters:

1 import json
2 import os
3 import fastapi
4 from fastapi.responses import StreamingResponse
5 from fastapi import Request
6 from openai import AsyncOpenAI
7 import uvicorn
8 import logging
9 from dotenv import load_dotenv
10 from pydantic import BaseModel
11 from typing import List, Optional
12 
13 # Load environment variables from .env file
14 load_dotenv()
15 
16 # Retrieve API key from environment
17 OPENAI_API_KEY = os.getenv('OPENAI_API_KEY')
18 if not OPENAI_API_KEY:
19     raise ValueError("OPENAI_API_KEY not found in environment variables")
20 
21 app = fastapi.FastAPI()
22 oai_client = AsyncOpenAI(api_key=OPENAI_API_KEY)
23 
24 class Message(BaseModel):
25     role: str
26     content: str
27 
28 class ChatCompletionRequest(BaseModel):
29     messages: List[Message]
30     model: str
31     temperature: Optional[float] = 0.7
32     max_tokens: Optional[int] = None
33     stream: Optional[bool] = False
34     user_id: Optional[str] = None
35     elevenlabs_extra_body: Optional[dict] = None
36 
37 @app.post("/v1/chat/completions")
38 async def create_chat_completion(request: ChatCompletionRequest) -> StreamingResponse:
39     oai_request = request.dict(exclude_none=True)
40     print(oai_request)
41     if "user_id" in oai_request:
42         oai_request["user"] = oai_request.pop("user_id")
43 
44     if "elevenlabs_extra_body" in oai_request:
45         oai_request.pop("elevenlabs_extra_body")
46 
47     chat_completion_coroutine = await oai_client.chat.completions.create(**oai_request)
48 
49     async def event_stream():
50         try:
51             async for chunk in chat_completion_coroutine:
52                 chunk_dict = chunk.model_dump()
53                 yield f"data: {json.dumps(chunk_dict)}\n\n"
54             yield "data: [DONE]\n\n"
55         except Exception as e:
56             logging.error("An error occurred: %s", str(e))
57             yield f"data: {json.dumps({'error': 'Internal error occurred!'})}\n\n"
58 
59     return StreamingResponse(event_stream(), media_type="text/event-stream")
60 
61 if __name__ == "__main__":
62     uvicorn.run(app, host="0.0.0.0", port=8013)

Example Request

With this custom message setup, your LLM will receive requests in this format:

1 {
2   "messages": [
3     {
4       "role": "system",
5       "content": "\n  <Redacted>"
6     },
7     {
8       "role": "assistant",
9       "content": "Hey I'm currently unavailable."
10     },
11     {
12       "role": "user",
13       "content": "Hey, who are you?"
14     }
15   ],
16   "model": "gpt-4o",
17   "temperature": 0.5,
18   "max_tokens": 5000,
19   "stream": true,
20   "elevenlabs_extra_body": {
21     "UUID": "123e4567-e89b-12d3-a456-426614174000",
22     "parameter-1": "value-1",
23     "parameter-2": "value-2"
24   }
25 }