How to dub video and audio with ElevenLabs
Learn how to automate the dubbing of audio and video files into various languages using the ElevenLabs API
Introduction
Dubbing videos and audio files from one language to another can be a great way to reach a wider audience. The ElevenLabs API provides a convenient way to automatically dub media files using state-of-the-art technology. In this guide, we will walk you through how to upload a video or audio file, dub it, and download the translated video. We’ll also discuss how to directly dub a link such as a YouTube, TikTok, or Twitter video.
If you’re looking to jump straight into the action, the complete code is available on the following repos:
On the 8th of May 2024 we launched the Dubbing API for all ElevenLabs tiers
How to upload and dub a video or audio file
Requirements
Before proceeding, please ensure that you have the following:
- An ElevenLabs account with an API key (here’s how to find your API key).
- Python or Node.js installed on your machine
Then, install the ElevenLabs SDK as shown below
Install the necessary packages to manage your environmental variables:
Next, create a .env
file in your project directory and fill it with your credentials like so:
ELEVENLABS_API_KEY=your_elevenlabs_api_key_here
Start the dubbing
First we want to send the file to the ElevenLabs dubbing service
Check for completion
The wait_for_dubbing_completion()
function within the dubbing_utils.py
file polls the API to check whether the dubbing process is complete. If completed, it proceeds to the next step; otherwise, it reports the status or failure.
Save the video locally
Upon completion of dubbing, the download_dubbed_file()
function in dubbing_utils.py
will save the dubbed file to a local directory, typically under the data/{dubbing_id}/{language_code}.mp4
.
Putting it together
We add the wait_for_dubbing_completion
(waitForDubbingCompletion
) function and the download_dubbed_file
(downloadDubbedFile
) function together to create the final function.
We then use the final the function as shown below.
How to dub a video from YouTube, TikTok, Twitter or Vimeo
For dubbing web-based content, instead of uploading a file you can pass in a URL. This supports popular platforms like YouTube, TikTok, Twitter, and Vimeo.
You can then call the function as shown below.
Conclusion
With this guide and the accompanying code structure, you now have a basic setup for dubbing audio and video content using the ElevenLabs API. Whether you’re working with local files or content from URLs, you can create multilingual versions of your media to cater to diverse audiences.
Remember to always follow the best practices when dealing with API keys and sensitive data, and consult the ElevenLabs API documentation for more advanced features and options. Happy dubbing!
For additional information on dubbing capabilities, translation services, and available languages, please refer to the ElevenLabs API documentation.
Should you encounter any issues or have questions, our GitHub Issues page is open for your queries and feedback.
List of supported languages for dubbing
No | Language Name | Language Code |
---|---|---|
1 | English | en |
2 | Hindi | hi |
3 | Portuguese | pt |
4 | Chinese | zh |
5 | Spanish | es |
6 | French | fr |
7 | German | de |
8 | Japanese | ja |
9 | Arabic | ar |
10 | Russian | ru |
11 | Korean | ko |
12 | Indonesian | id |
13 | Italian | it |
14 | Dutch | nl |
15 | Turkish | tr |
16 | Polish | pl |
17 | Swedish | sv |
18 | Filipino | fil |
19 | Malay | ms |
20 | Romanian | ro |
21 | Ukrainian | uk |
22 | Greek | el |
23 | Czech | cs |
24 | Danish | da |
25 | Finnish | fi |
26 | Bulgarian | bg |
27 | Croatian | hr |
28 | Slovak | sk |
29 | Tamil | ta |