# Did Translation Models Get More Robust Without Anyone Even Noticing?

arXiv: https://arxiv.org/abs/2403.03923 · HTML: https://mlsanity.com/paper/2403.03923/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 54 references that are in the map.
- First posted on arXiv 6 Mar 2024.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [No Language Left Behind: Scaling Human-Centered Machine Translation](https://mlsanity.com/paper/2207.04672/) (2022)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [How Good Are GPT Models at Machine Translation? A Comprehensive Evaluation](https://mlsanity.com/paper/2302.09210/) (2023)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Language Model Tokenizers Introduce Unfairness Between Languages](https://mlsanity.com/paper/2305.15425/) (2023)
- [CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2209.06243/) (2022)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://mlsanity.com/paper/2402.17733/) (2024)
- [Marian: Fast Neural Machine Translation in C++](https://mlsanity.com/paper/1804.00344/) (2018)
- [Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLP](https://mlsanity.com/paper/2112.10508/) (2021)
- [The Tatoeba Translation Challenge -- Realistic Data Sets for Low Resource and Multilingual MT](https://mlsanity.com/paper/2010.06354/) (2020)
- [On Adversarial Examples for Character-Level Neural Machine Translation](https://mlsanity.com/paper/1806.09030/) (2018)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018)
- [MTNT: A Testbed for Machine Translation of Noisy Text](https://mlsanity.com/paper/1809.00388/) (2018)
- [Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation](https://mlsanity.com/paper/1902.01509/) (2019)
- [Steering Large Language Models for Machine Translation with Finetuning and In-Context Learning](https://mlsanity.com/paper/2310.13448/) (2023)
- [Robust Open-Vocabulary Translation from Visual Text Representations](https://mlsanity.com/paper/2104.08211/) (2021)
- [Improving Robustness of Machine Translation with Synthetic Noise](https://mlsanity.com/paper/1902.09508/) (2019)
- [NeuSpell: A Neural Spelling Correction Toolkit](https://mlsanity.com/paper/2010.11085/) (2020)
- [Unnatural Error Correction: GPT-4 Can Almost Perfectly Handle Unnatural Scrambled Text](https://mlsanity.com/paper/2311.18805/) (2023)
- [An Analysis of Source-Side Grammatical Errors in NMT](https://mlsanity.com/paper/1905.10024/) (2019)
- [Adversarial Subword Regularization for Robust Neural Machine Translation](https://mlsanity.com/paper/2004.14109/) (2020)

## Built on by (most cited first, 9 of 9)

- [Modeling User Preferences with Automatic Metrics: Creating a High-Quality Preference Dataset for Machine Translation](https://mlsanity.com/paper/2410.07779/) (2024)
- [From TOWER to SPIRE: Adding the Speech Modality to a Translation-Specialist LLM](https://mlsanity.com/paper/2503.10620/) (2025)
- [From SALAMANDRA to SALAMANDRATA: BSC Submission for WMT25 General Machine Translation Shared Task](https://mlsanity.com/paper/2508.12774/) (2025)
- [Translation Analytics for Freelancers: I. Introduction, Data Preparation, Baseline Evaluations](https://mlsanity.com/paper/2504.14619/) (2025)
- [How Important is `Perfect' English for Machine Translation Prompts?](https://mlsanity.com/paper/2507.09509/) (2025)
- [MT-LENS: An all-in-one Toolkit for Better Machine Translation Evaluation](https://mlsanity.com/paper/2412.11615/) (2024)
- [When the Gold Standard Isn't Necessarily Standard: Challenges of Evaluating the Translation of User-Generated Content](https://mlsanity.com/paper/2512.17738/) (2025)
- [Pre-Editorial Normalization for Automatically Transcribed Medieval Manuscripts in Old French and Latin](https://mlsanity.com/paper/2602.13905/) (2026)
- [Translation as a Computationally Efficient Bridge: Feasibility of English BERT for Low-Resource Languages](https://mlsanity.com/paper/2607.12612/) (2026)

## Abstract (arXiv)

Neural machine translation (MT) models achieve strong results across a variety of settings, but it is widely believed that they are highly sensitive to "noisy" inputs, such as spelling errors, abbreviations, and other formatting issues. In this paper, we revisit this insight in light of recent multilingual MT models and large language models (LLMs) applied to machine translation. Somewhat surprisingly, we show through controlled experiments that these models are far more robust to many kinds of noise than previous models, even when they perform similarly on clean data. This is notable because, even though LLMs have more parameters and more complex training processes than past models, none of the open ones we consider use any techniques specifically designed to encourage robustness. Next, we show that similar trends hold for social media translation experiments -- LLMs are more robust to social media text. We include an analysis of the circumstances in which source correction techniques can be used to mitigate the effects of noise. Altogether, we show that robustness to many types of noise has increased.