Conversational AI
- Default LLM update: Changed the default agent LLM from Gemini 1.5 Flash to Gemini 2.0 Flash for improved performance.
- Fixed incorrect conversation abandons: Improved detection of conversation continuations, preventing premature abandons when users repeat themselves.
- Twilio information in history: Added Twilio call details to conversation history for better tracking.
- Knowledge base redesign: Redesigned the knowledge base interface.
- System dynamic variables: Added system dynamic variables to use time, conversation id, caller id and other system values as dynamic variables in prompts and tools.
- Twilio client initialisation: Adds an agent level override for conversation initiation client data twilio webhook.
- RAG chunks in history: Added retrieved chunks by RAG to the call transcripts in the history view.
Speech to Text
- Reduced pricing: Reduced the pricing of our Scribe model, see more here.
- Improved VAD detection: Enhanced Voice Activity Detection with better pause detection at segment boundaries and improved handling of silent segments.
- Enhanced diarization: Improved speaker clustering with a better ECAPA model, symmetric connectivity matrix, and more selective speaker embedding generation.
- Fixed ASR bugs: Resolved issues with VAD rounding, silence and clustering that affected transcription accuracy.
Studio
- Disable publishing UI: Added ability to disable the publishing interface for specific workspace members to support enterprise workflows.
- Snapshot API improvement: Modified endpoints for project and chapter snapshots to return an empty list instead of throwing errors when snapshots can’t be downloaded.
- Disabled auto-moderation: Turned off automatic moderation based on Text to Speech generations in Studio.
Workspaces
- Fixed API key editing: Resolved an issue where editing workspace API keys would reset character limits to zero, causing the keys to stop working.
- Optimized free subscriptions: Fixed an issue with refreshing free subscription character limits,
API
View API changes
New Endpoints
- Added 3 new endpoints:
Updated Endpoints
Dubbing
- Updated Dubbing endpoints:
- Dub a video or audio file - Added
use_replacement_voices_from_library
property and madesource_path
,target_language
,source_language
nullable - Resource dubbing - Made
language_codes
array nullable - Add language to dubbing resource - Made
language_code
nullable - Add speaker segment - Made
text
nullable - Translate dubbing resource - Made
target_languages
array nullable - Update dubbing segment - Made
start_time
andend_time
nullable
- Dub a video or audio file - Added
Project Management
- Updated Project endpoints:
- Add project - Made
metadata
,project_name
,description
nullable - Create podcast - Made
title
,description
,author
nullable - Get project - Made
last_modified_at
,created_at
,project_name
nullable - Add chapter - Made
chapter_id
,word_count
,statistics
nullable - Update chapter - Made
content
andblocks
properties nullable
- Add project - Made
Conversational AI
- Updated Conversational AI endpoints:
- Update agent - Made
conversation_config
,platform_settings
nullable and addedworkspace_overrides
property - Create agent - Made
agent_name
,prompt
,widget_config
nullable and addedworkspace_overrides
property - Add to knowledge base - Made
document_name
nullable - Get conversation - Added
twilio_call_data
model and madetranscript
,metadata
nullable
- Update agent - Made
Text to Speech
- Updated Text to Speech endpoints:
- Convert text to speech - Made
voice_settings
,text_input
nullable and deprecateduse_pvc_as_ivc
property - Stream text to speech - Made
voice_settings
,text_input
nullable and deprecateduse_pvc_as_ivc
property - Convert with timestamps - Made
character_alignment
andword_alignment
nullable
- Convert text to speech - Made
Voice Management
- Updated Voice endpoints:
- Create voice previews - Added
loudness
,quality
,guidance_scale
properties - Create voice from preview - Added
speaker_separation
properties and madevoice_id
,name
,labels
nullable - Get voice - Added
speaker_boost
,speaker_clarity
,speaker_isolation
properties
- Create voice previews - Added
Speech to Text
- Updated Speech to Text endpoint:
- Convert speech to text - Added
biased_keywords
property
- Convert speech to text - Added
Other Updates
- Download history - Added application/zip content type and 400 response
- Add pronunciation dictionary from file - Made
dictionary_name
anddescription
nullable