YouTube integrates AI-powered dubbing tool

YouTube is currently testing a new tool that will help creators automatically dub their videos into other languages using AI, the company announced Thursday at VidCon. YouTube teamed up with AI-powered dubbing service Aloud, which is part of Google’s in-house incubator Area 120.

Earlier this year, YouTube introduced support for multi-language audio tracks, which allows creators to add dubbing to their new and existing videos, letting them reach a wider international audience. As of June 2023, creators have dubbed more than 10,000 videos in over 70 languages, the company told TechCrunch.

Previously, creators had to partner directly with third-party dubbing providers to create their audio tracks, which can be a time-consuming and expensive process. Aloud lets them dub videos at no additional cost.

Google first introduced Aloud in 2022. The AI-powered dubbing product transcribes a video for the creator then translates and produces a dubbed version. Creators can review and edit the transcription before Aloud generates the dub.

YouTube is testing the tool with hundreds of creators, YouTube’s VP of Creator Products, Amjad Hanif, said to the crowd yesterday. Soon, the company will open the tool to all creators. Aloud is currently only available in English, Spanish and Portuguese. However, there will be more languages offered in the future, such as Hindi and Bahasa-Indonesia, among others.

Hanif added that YouTube is “working to make translated audio tracks sound like the creator’s voice, with more expression and lip sync.” YouTube confirmed to TechCrunch that, in the future, generative AI would allow Aloud to launch features like voice preservation, better emotion transfer and lip reanimation.