CUSTOMIZATIONLLMCustom LLM

SambaNova Cloud

Connect an agent to a custom LLM on SambaNova Cloud.

Overview

SambaNova Cloud is the fastest provider of the best open source models, including DeepSeek R1, DeepSeek V3, Llama 4 Maverick and others. Through an OpenAI-compatible API endpoint, you can set up your Conversational AI agent on ElevenLabs in a just few minutes.

Watch this video for a walkthrough and demo of how you can configure your ElevenLabs Conversational AI agent to leverage SambaNova’s blazing-fast LLMs!

Choosing a model

To make use of the full power of ElevenLabs Conversational AI you need to use a model that supports tool use and structured outputs. SambaNova recommends the following models for their accuracy and performance:

  • DeepSeek-V3-0324 (671B model)
  • Meta-Llama-3.3-70B-Instruct
  • Llama-4-Maverick-17B-128E-Instruct
  • Qwen3-32B

For up-to-date information on model-specific context windows, please refer to this page.

Note that Meta-Llama-3.3-70B-Instruct is SambaNova’s most battle-tested model. If any model is causing issues, you may report it on SambaNova’s Community page.

Configuring your ElevenLabs agent with a SambaNova LLM

1

Navigate to cloud.sambanova.ai/apis and create a new API key.

Add Secret

2

Once you have your API key, you can test it by running the following curl command:

$curl -H "Authorization: Bearer <your-api-key>" \
> -H "Content-Type: application/json" \
> -d '{
>"stream": true,
>"model": "DeepSeek-V3-0324",
>"messages": [
>	{
>		"role": "system",
>		"content": "You are a helpful assistant"
>	},
>	{
>		"role": "user",
>		"content": "Hello"
>	}
>]
>}' \
> -X POST https://api.sambanova.ai/v1/chat/completions
3

Create a new AI Agent or edit an existing one.

4

Scroll down to the “Workspace Secrets” section and select “Add Secret”. Name the key SAMBANOVA_API_KEY and copy the value from the SambaNova Cloud dashboard. Be sure to hit “Save” to make the secret available to your agent.

Add Secret

5

Choose “Custom LLM” from the dropdown menu.

Choose custom llm

6

For the Server URL, specify SambaNova’s OpenAI-compatible API endpoint: https://api.sambanova.ai/v1. For the Model ID, specify one the model names indicated above (e.g., Meta-Llama-3.3-70B-Instruct) and select the SAMBANOVA_API_KEY API key from the dropdown menu.

Enter url

7

Set the max tokens to 1024 to restrict the agent’s output for brevity. Also be sure to include an instruction in the System Prompt for the model to respond in 500 words or less.

Enter url

8

Save your changes and click on “Test AI Agent” to chat with your SambaNova-powered agent!