Meta introduces SeamlessM4T, an AI model that translates and transcribes nearly 100 languages
A universal translator is something we may expect to see in a Star Wars film, but thanks to the developments and advances in AI, such a thing may soon be reality. Social media giant Meta has been intensifying its focus on AI models in recent times, and now, it has released a new speech-to-text model that can translate nearly 100 languages. Known as Seamless M4T, the AI model can translate and transcribe close to 100 languages across text and speech, and is available in open source along with SeamlessAlign, a new translation data set.
Meta’s SeamlessM4T, which stands for Massively Multilingual and Multimodal Machine Translation, emerges as a new innovation in the realm of AI-powered language translation. Going beyond conventional translation tools, SeamlessM4T positions itself as a solution that handles a spectrum of language-related tasks, including speech recognition, speech-to-text translation, text-to-speech translation, speech-to-speech translation, and text-to-text translation.
As mentioned earlier, Seamless M4T recognizes 100 input languages for speech-to-speech (and converts them into 36 output languages, including English), and text-to-speech actions (and converts them into 35 output languages).
“SeamlessM4T builds on advancements we and others have made over the years in the quest to create a universal translator. Last year, we released No Language Left Behind (NLLB), a text-to-text machine translation model that supports 200 languages, and has since been integrated into Wikipedia as one of the translation providers,” Meta wrote in a blog post. SeamlessM4T is currently being released under a research license.
“We also shared a demo of our Universal Speech Translator, which was the first direct speech-to-speech translation system for Hokkien, a language without a widely used writing system. And earlier this year, we revealed Massively Multilingual Speech, which provides speech recognition, language identification and speech synthesis technology across more than 1,100 languages,” the company added.
Unlike its predecessors, which often relied on multiple intermediary steps, SeamlessM4T streamlines the translation process, minimizing errors and boosting efficiency. This achievement propels the AI community closer to the elusive goal of a universal translator, something that has been something out of a sci-fi novel or film so far.
Meta’s SeamlessM4T offers a glimpse into a future where language is no longer a barrier to communication. By consolidating various translation tasks into a single powerhouse and catering to an extensive array of languages, this innovation could transform the way we interact globally, and Meta claims that SeamlessM4T represents a “significant breakthrough” in the field of AI-powered speech-to-speech and speech-to-text. The company added that SeamlessM4T can also detect code-switching, a linguistic phenomenon where speakers fluidly transition between languages in a single sentence.
