Together AI

Connect an agent to a custom LLM on Together AI.

Overview

Together AI provides an AI Acceleration Cloud, allowing you to train, fine-tune, and run inference on AI models blazing fast, at low cost, and at production scale.

Instantly run 200+ models including DeepSeek, Llama3, Mixtral, and Stable Diffusion, optimized for peak latency, throughput, and context length.

Choosing a model

To make use of the full power of ElevenLabs Agents you need to use a model that supports tool use and structured outputs. Together AI supports function calling for these models:

meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo

meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo

meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo

meta-llama/Llama-3.3-70B-Instruct-Turbo

mistralai/Mixtral-8x7B-Instruct-v0.1

mistralai/Mistral-7B-Instruct-v0.1

With this in mind, it’s recommended to use at least meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo for your ElevenLabs Agents agent.

Set up Llama 3.1 on Together AI

Navigate to api.together.xyz/settings/api-keys and create a new API key.

Once you have your API key, you can test it by running the following curl command:

$ curl https://api.together.xyz/v1/chat/completions -s \
> -H "Content-Type: application/json" \
> -H "Authorization: Bearer <API_KEY>" \
> -d '{
> "model": "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
> "messages": [{
>     "role": "user",
>     "content": "Hello, how are you?"
> }]
> }'

Navigate to your AI Agent, scroll down to the “Secrets” section and select “Add Secret”. After adding the secret, make sure to hit “Save” to make the secret available to your agent.

Choose “Custom LLM” from the dropdown menu.

For the Server URL, specify Together AI’s OpenAI-compatible API endpoint: https://api.together.xyz/v1. For the Model ID, specify meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo as discussed above, and select your API key from the dropdown menu.

Now you can go ahead and click “Test AI Agent” to chat with your custom Llama 3.1 model.

$	curl https://api.together.xyz/v1/chat/completions -s \
>	-H "Content-Type: application/json" \
>	-H "Authorization: Bearer <API_KEY>" \
>	-d '{
>	"model": "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
>	"messages": [{
>	"role": "user",
>	"content": "Hello, how are you?"
>	}]
>	}'