Realtime
Realtime speech-to-text transcription service. This WebSocket API enables streaming audio input and receiving transcription results.
Event Flow
- Audio chunks are sent as
input_audio_chunkmessages - Transcription results are streamed back as
partial_transcript(interim) andcommitted_transcript(stable/final for that segment) - Supports manual commit (the default), VAD-based automatic commit, or turn prediction for models that predict turns
Authentication is done either by providing a valid API key in the xi-api-key header or by providing a valid token in the token query parameter. Tokens can be generated from the single use token endpoint. Use tokens if you want to transcribe audio from the client side.
Handshake
Headers
Query parameters
The ID of the model to use for speech-to-text transcription.
Single use token for authentication. Only used when initiating a session from the client. If provided, xi-api-key is no longer required for authentication.
The encoding format of the audio. Supported formats: pcm_8000, pcm_16000, pcm_22050, pcm_24000, pcm_44100, pcm_48000, ulaw_8000.
An ISO-639-1 or ISO-639-3 language_code corresponding to the language of the audio file. Can sometimes improve transcription performance if known beforehand. Defaults to null, in this case the language is predicted automatically.
Additional ISO-639-1 or ISO-639-3 language codes that may be present in the audio. Providing them makes language identification more reliable by only focusing on a certain set of languages. Each code is validated the same way as language_code.
Commit strategy for speech segmentation. ‘manual’ (the default for all models) requires explicit commits, ‘vad’ automatically segments speech using silence detection, ‘turn_prediction’ additionally commits as soon as the model predicts the speaker has finished their turn (only for models that predict turns).
Enable word/character-level timestamps in a delayed committed_transcript_with_timestamps message. When enabled, you’ll receive an additional message with timestamps after each commit. Default: false.
Enable language detection in a delayed committed_transcript_with_timestamps message. When enabled, you’ll receive an additional message with detected language_code after each commit. Default: false.
List of keyterms to bias the model towards. Maximum 50 keyterms. Adds a 20% premium to the base transcription cost.
Transcribe every language spoken in its own script, including code-switching mid-utterance. Scribe v3 Live models only.
Add the words of every partial_transcript, as in batch diarization, each with a speaker_id (speaker_0, speaker_1, …) once the speaker is identified, null until then; they update as more audio is heard. After each committed_transcript, which never waits for them, a diarized_transcript carries its words with their final speakers. scribe_v3_live only, and cannot be combined with filter_background_audio. Default: false.
Detect entities on committed transcripts. Can be ‘all’, a single entity type or category, or a list of types/categories (‘pii’, ‘phi’, ‘pci’, ‘other’, ‘offensive_language’). When enabled, detected entities are delivered in a separate ‘committed_transcript_entities’ event with their text, type, and character positions.
Redact entities from committed transcripts. Accepts the same format as entity_detection: ‘all’, a category (‘pii’, ‘phi’, ‘pci’, ‘other’, ‘offensive_language’), or specific entity types. The redacted text is delivered in a separate ‘redacted_transcript’ event after the committed transcript. If entity detection fails, an ‘error’ event is sent and the connection is closed. Cannot be combined with transcript_edit.
How to format redacted entities. ‘redacted’ replaces with {REDACTED}, ‘entity_type’ replaces with {ENTITY_TYPE}, ‘enumerated_entity_type’ replaces with {ENTITY_TYPE_N} where N enumerates each occurrence. Only used when entity_redaction is set.
Natural-language instruction applied to each committed transcript (max 2000 characters). The edited text is delivered in a separate ‘edited_transcript’ event containing the committed text and the edited text. Cannot be combined with entity_detection or entity_redaction. Adds a 30% premium to the base transcription cost, billed for at least 10 seconds of audio per committed transcript.
Enable background speech filtering to reduce false activations from nearby conversations and ambient noise. When enabled without an explicit vad_threshold, a lower default threshold is applied. Cannot be combined with diarize.
Opt-in keepalive interval in milliseconds (500-10000). While the audio being streamed contains no speech, the server sends a partial_transcript about this often, so clients that enforce a read timeout on incoming frames do not drop the connection during long pauses. The keepalive has empty text (text: ""), or repeats the latest partial text if the current segment is not committed yet. Keepalives are only sent after the transcription models have processed silent audio, so audio must keep streaming. Cadence is rounded to the audio processing granularity (about one second). Disabled by default.
When enable_logging is set to false zero retention mode will be used for the request. This will mean history features are unavailable for this request, including request stitching. Zero retention mode may only be used by enterprise customers.
Send
Receive
Final words and speaker IDs for a committed segment when diarize is enabled. Sent after committed_transcript, in commit order; correlate the messages by segment_index.
Two speaker IDs have been identified as the same person. Replace earlier uses of from_speaker_id with to_speaker_id, applying merges in the order received. Only sent when diarize is enabled.
Committed transcription result with word-level timestamps.
Detected entities for a committed transcript. Only sent when the entity_detection query parameter is set.
Committed transcript with entities redacted, sent after the committed_transcript when entity_redaction is set.
Edited version of a committed transcript, delivered as a separate event when the transcript_edit query parameter is set. text is the committed text the instruction was applied to (use it to correlate with the committed_transcript event, since edits may arrive out of commit order). If no edits were made, edited_text is identical to text.
Non-fatal notice. Sent after session_started when enable_logging=false was requested but zero retention mode was not applied; the session continues and is still logged.
The connection parameters were rejected; the session is closed afterwards.