With the vision of creating a “universal translator”, the open-source Meta AI model “NLLB-200” can translate 200 languages

Language is also described as a mode of communication used by human beings as members of a social group, as participants of a cultural group to express themselves. But people run into problems at some point when they want to read content, watch a movie, or engage in conversation with people. However, they found it difficult to engage with it simply because people don’t know the language in question. This is especially true for the hundreds of millions of people who speak many languages ​​in Africa and Asia.

To overcome this problem, Meta announced its high-quality machine translation capability model for translating most of the world’s languages ​​called NLLB (No Language Left Behind). NLLB-200 is an effort to develop an AI model of single-language translation by meta-researchers that could translate up to 200 languages ​​(many of which are still unsupported by even some of the best models in existence today ) with peak results. Fewer than 25 African languages ​​are supported by language translation tools widely used today, while NLLB-200 increases that number to 55 languages, including increased accuracy of up to 70% for some of them. Comparing the translation quality to previous AI research, NLLB-200 achieves an average score of 44% in the 10,000 directions of the FLORES-101 benchmark, providing increased accuracy of up to 70% for some of the Asian languages and regional Africans. .

Meta has partnered with Wikimedia Foundation, the non-profit organization that hosts Wikipedia and other free knowledge projects to provide access to the information it shares. Most of the articles he shares are available in English, creating a disparity between articles in other languages. Now, Wikipedia uses NLLB to translate its articles into 20 different low-resource languages, 10 of which were previously not supported by any language translation tool.

Meta hinted that NLLB’s search advancements will support more than 25 billion translations served daily across Facebook News Feed, Instagram and our other platforms. Accurate, high-quality translations into more languages ​​would help identify harmful content and misinformation, protect the integrity of elections, and combat online sexual exploitation and human trafficking on these platforms . Additionally, to help other developers and researchers improve their translation tools and contribute to the model, Meta announced the open source of this model along with the model source code and training dataset. He also announced grants of up to $200,000 for impactful uses of the NLLB-200 to researchers and nonprofits with initiatives focused on sustainability, food security, gender-based violence, education, or other areas supporting the United Nations Sustainable Development Goals.

Meta first presented its preliminary model, M2M-100, which could translate up to 100 languages ​​in 2020. To expand this capability to another 100 languages, Meta tried to incorporate new methods for acquiring training data and ideas to evolve the model without compromising its performance. , avoid overfitting or underfitting, and evaluate and improve outcomes. For the training datasets, Meta tried to leverage LASER3 (a toolkit developed and enhanced by meta, which is a seamless transfer into NLP) instead of LSTM, a newer version. LASER3 uses a self-supervising trained transformer model with a masked language modeling objective. This is also open-source by Meta if you want to watch it. After collecting highly accurate parallel texts in different languages, Meta researchers faced significant challenges in extending this model from 100 to 200 languages. For more low-resource language pairs in the training data, the model started to overfit while training it for long periods of time. To overcome these problems, innovation has been made on three fronts: curriculum regularization and learning, self-supervised learning, and back-translation diversification. Once all these elements were completed, the model was trained on the new Research Supercluster (RSC)among the fastest AI supercomputers in the world, with 54B parameters.

With all of this, as the metaverse begins to take shape, this model of Meta would help transform language translation capabilities in various fields. For example, language translations, subtitles, multimedia, etc., and the ability to create technologies that work well in a wider range of languages ​​will help democratize access to immersive experiences in virtual worlds.

References:

• https://ai.facebook.com/blog/nllb-200-high-quality-machine-translation/
• https://about.fb.com/news/2022/07/new-meta-ai-model-translates-200-languages-making-technology-more-accessible/
• https://www.ithome.com.tw/news/151819
• https://www.theverge.com/2022/7/6/23194241/meta-facebook-ai-universal-translation-project-no-language-left-behind-open-source-model
• https://www.zdnet.com/article/metas-latest-ai-model-will-make-content-available-in-hundreds-of-languages/
• https://venturebeat.com/2022/07/06/metas-open-source-ai-model-leaves-no-language-behind/

Please Don't Forget To Join Our ML Subreddit