Facebook performs some 4.5 billion automatic translations everyday. The social networking site previously used simpler phrase-based machine translation models, but it’s now switched to the more advanced method. As of yesterday, the automatic translations are processed using neural networks. Facebook stated in a Blog post that “Creating seamless, highly accurate translation experiences for the 2 billion people who use Facebook is difficult”. Facebook equally pointed out some difficulties (translations of slangs, abbrevations etc.) they experienced using the phrase-based machine translation model and thought of a solution for it. “We need to account for context, slang, typos, abbreviations, and intent simultaneously.” They added, in that same blog post
The big difference between the old system and the new one is the attention span. While the phrase-based system translated sentences word by word, or by looking at short phrases, the neural networks consider whole sentences at a time. This is achieved by using a particular sort of machine learning component known as an LSTM which is short for “long short-term memory” network.
Compare these two examples from Facebook of a Turkish-to-English translation. The top one represents the old phrase-based system, and the bottom one represents the new system. This illustration would show you how taking into account the full context of the sentence produces a more accurate result.
“With the new system, we saw an average relative increase of 11 percent in BLEU — a widely used metric for judging the accuracy of machine translation — across all languages compared with the phrase-based systems” Facebook said.
The neural system generates a placeholder for the unknown word when a word in a sentence doesn’t have a direct corresponding translation in a target language. A translation of that word is searched for in a sort of in-house dictionary built from Facebook’s training data, and the unknown word is replaced. That allows abbreviations like “grt” to be translated into their intended meaning — “great.”
“Neural networks open up many future development paths related to adding further context, such as a photo accompanying the text of a post, to create better translations,” the company said. “We are also starting to explore multilingual models that can translate many different language directions.”