Optimizing LLM costs

Practical strategies to reduce LLM inference expenses on the ElevenLabs platform.

Overview

Managing Large Language Model (LLM) inference costs is essential for developing sustainable AI applications. This guide outlines key strategies to optimize expenditure on the ElevenLabs platform by effectively utilizing its features. For detailed model capabilities and pricing, refer to our main LLM documentation.

ElevenLabs supports reducing costs by reducing inference of the models during periods of silence. These periods are billed at 5% of the usual per minute rate. See the ElevenAgents overview page for more details.

Understanding inference costs

LLM inference costs on our platform are primarily influenced by:

Input tokens: The amount of data processed from your prompt, including user queries, system instructions, and any contextual data.
Output tokens: The number of tokens generated by the LLM in its response.
Model choice: Different LLMs have varying per-token pricing. More powerful models generally incur higher costs.

Monitoring your usage via the ElevenLabs dashboard or API is crucial for identifying areas for cost reduction.

Strategic model selection

Choosing the most appropriate LLM is a primary factor in cost efficiency.

Right-sizing: Select the least complex (and typically less expensive) model that can reliably perform your specific task. Avoid using high-cost models for simple operations. For instance, models like Google’s gemini-2.0-flash offer highly competitive pricing for many common tasks. Always cross-reference with the full Supported LLMs list for the latest pricing and capabilities.
Experimentation: Test various models for your tasks, comparing output quality against incurred costs. Consider language support, context window needs, and specialized skills.

Prompt optimization

Prompt engineering is a powerful technique for reducing token consumption and associated costs. By crafting clear, concise, and unambiguous system prompts, you can guide the model to produce more efficient responses. Eliminate redundant wording and unnecessary context that might inflate your token count. Consider explicitly instructing the model on your desired output length—for example, by adding phrases like “Limit your response to two sentences” or “Provide a brief summary.” These simple directives can significantly reduce the number of output tokens while maintaining the quality and relevance of the generated content.

Modular design: For complex conversational flows, leverage agent-agent transfer. This allows you to break down a single, large system prompt into multiple, smaller, and more specialized prompts, each handled by a different agent. This significantly reduces the token count per interaction by loading only the contextually relevant prompt for the current stage of the conversation, rather than a comprehensive prompt designed for all possibilities.

Leveraging knowledge and retrieval

For applications requiring access to large information volumes, Retrieval Augmented Generation (RAG) and a well-maintained knowledge base are key.

Efficient RAG:
- RAG reduces input tokens by providing the LLM with only relevant snippets from your Knowledge Base, instead of including extensive data in the prompt.
- Optimize the retriever to fetch only the most pertinent “chunks” of information.
- Fine-tune chunk size and overlap for a balance between context and token count.
- Learn more about implementing RAG.
Context size:
- Ensure your Knowledge Base contains accurate, up-to-date, and relevant information.
- Well-structured content improves retrieval precision and reduces token usage from irrelevant context.

Intelligent tool utilization

Using Server Tools allows LLMs to delegate tasks to external APIs or custom code, which can be more cost-effective.

Task offloading: Identify deterministic tasks, those requiring real-time data, complex calculations, or API interactions (e.g., database lookups, external service calls).
Orchestration: The LLM acts as an orchestrator, making structured tool calls. This is often far more token-efficient than attempting complex tasks via prompting alone.
Tool descriptions: Provide clear, concise descriptions for each tool, enabling the LLM to use them efficiently and accurately.

Checklist

Consider applying these techniques to reduce cost:

Feature	Cost impact	Action items
LLM choice	Reduces per-token cost	Select the smallest, most economical model that reliably performs the task. Experiment and compare cost vs. quality.
Custom LLMs	Potentially lower inference cost for specialized tasks	Evaluate for high-volume, specific tasks; fine-tune on proprietary data to create smaller, efficient models.
System prompts	Reduces input & output tokens, guides model behavior	Be concise, clear, and specific. Instruct on desired output format and length (e.g., “be brief,” “use JSON”).
User prompts	Reduces input tokens	Encourage specific queries; use few-shot examples strategically; summarize or select relevant history.
Output control	Reduces output tokens	Prompt for summaries or key info; use `max_tokens` cautiously; iterate on prompts to achieve natural conciseness.
RAG	Reduces input tokens by avoiding large context in prompt	Optimize retriever for relevance; fine-tune chunk size/overlap; ensure high-quality embeddings and search algorithms.
Knowledge base	Improves RAG efficiency, reducing irrelevant tokens	Curate regularly; remove outdated info; ensure good structure, metadata, and tagging for precise retrieval.
Tools (functions)	Avoids LLM calls for specific tasks; reduces tokens	Delegate deterministic, calculation-heavy, or external API tasks to tools. Design clear tool descriptions for the LLM.
Agent transfer	Enables use of cheaper models for simpler parts of tasks	Use simpler/cheaper agents for initial triage/FAQs; transfer to capable agents only when needed; decompose large prompts into smaller prompts across various agents

Conversation history management

For stateful conversations, rather than passing in multiple conversation transcripts as a part of the system prompt, implement history summarization or sliding window techniques to keep context lean. This can be particularly effective when building consumer applications and can often be managed upon receiving a post-call webhook.

Continuously monitor your LLM usage and costs. Regularly review and refine your prompts, RAG configurations, and tool integrations to ensure ongoing cost-effectiveness.