“Meta has done a great job having a breadth of different things they support, like text-to-speech, speech-to-text, even automatic speech recognition,” says Chetan Jaiswal, a professor of laptop science at Quinnipiac University, who was not concerned within the analysis. “The mere number of languages they are supporting is a tremendous achievement.”
Human translators are nonetheless a significant a part of the interpretation course of, the researchers say within the paper, as a result of they can grapple with numerous cultural contexts and ensure the identical that means is conveyed from one language into one other. This step is essential, says Lynne Bowker, Canada Research Chair in Translation, Technologies and Society at Université Laval in Quebec, who didn’t work on Seamless. “Languages are a reflection of cultures, and cultures have their own ways of knowing things,” she says.
When it involves purposes like drugs or regulation, machine translations must be totally checked by a human, she says. If not, misunderstandings can end result. For instance, when Google Translate was used to translate public well being details about the covid-19 vaccine from the Virginia Department of Health in January 2021, it translated “not mandatory” in English into “not necessary” in Spanish, altering the entire that means of the message.
AI fashions have a lot more examples to coach on in some languages than others. This means present speech-to-speech fashions might be able to translate a language like Greek into English, the place there could also be many examples, however can’t translate from Swahili to Greek. The workforce behind Seamless aimed to resolve this downside by pre-training the model on thousands and thousands of hours of spoken audio in several languages. This pre-training allowed it to acknowledge normal patterns in language, making it simpler to course of much less extensively spoken languages as a result of it already had some baseline for what spoken language is meant to sound like.
The system is open-source, which the researchers hope will encourage others to construct upon its present capabilities. But some are skeptical of how helpful it could be in contrast with accessible options. “Google’s translation model is not as open-source as Seamless, but it’s way more responsive and fast, and it doesn’t cost anything as an academic,” says Jaiswal.
The most fun factor about Meta’s system is that it factors to the potential for immediate interpretation throughout languages within the not-too-distant future—just like the Babel fish in Douglas Adams’ cult novel The Hitchhiker’s Guide to the Galaxy. SeamlessM4T is quicker than current fashions however nonetheless not immediate. That mentioned, Meta claims to have a more moderen model of Seamless that’s as quick as human interpreters.
“While having this kind of delayed translation is okay and useful, I think simultaneous translation will be even more useful,” says Kenny Zhu, director of the Arlington Computational Linguistics Lab on the University of Texas at Arlington, who will not be affiliated with the new analysis.