Together AI

Connect an agent to a custom LLM on Together AI.

Overview

Together AI provides an AI Acceleration Cloud, allowing you to train, fine-tune, and run inference on AI models blazing fast, at low cost, and at production scale.

Instantly run 200+ models including DeepSeek, Llama3, Mixtral, and Stable Diffusion, optimized for peak latency, throughput, and context length.

Choosing a model

To make use of the full power of ElevenLabs Conversational AI you need to use a model that supports tool use and structured outputs. Together AI supports function calling for these models:

  • meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo
  • meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo
  • meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo
  • meta-llama/Llama-3.3-70B-Instruct-Turbo
  • mistralai/Mixtral-8x7B-Instruct-v0.1
  • mistralai/Mistral-7B-Instruct-v0.1

With this in mind, it’s recommended to use at least meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo for your ElevenLabs Conversational AI agent.

Set up Llama 3.1 on Together AI

1

Navigate to api.together.xyz/settings/api-keys and create a new API key.

Add Secret

2

Once you have your API key, you can test it by running the following curl command:

$curl https://api.together.xyz/v1/chat/completions -s \
>-H "Content-Type: application/json" \
>-H "Authorization: Bearer <API_KEY>" \
>-d '{
>"model": "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo",
>"messages": [{
>    "role": "user",
>    "content": "Hello, how are you?"
>}]
>}'
3

Navigate to your AI Agent, scroll down to the “Secrets” section and select “Add Secret”. After adding the secret, make sure to hit “Save” to make the secret available to your agent.

Add Secret

4

Choose “Custom LLM” from the dropdown menu.

Choose custom llm

5

For the Server URL, specify Together AI’s OpenAI-compatible API endpoint: https://api.together.xyz/v1. For the Model ID, specify meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo as discussed above, and select your API key from the dropdown menu.

Enter url

6

Now you can go ahead and click “Test AI Agent” to chat with your custom Llama 3.1 model.

Built with