# Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation

arXiv: https://arxiv.org/abs/1902.01509 · HTML: https://mlsanity.com/paper/1902.01509/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Feb 2019.

## Built on by (most cited first, 30 of 52)

- [Hallucinations in Large Multilingual Translation Models](https://mlsanity.com/paper/2303.16104/) (2023)
- [Measure and Improve Robustness in NLP Models: A Survey](https://mlsanity.com/paper/2112.08313/) (2021)
- [Why Should Adversarial Perturbations be Imperceptible? Rethink the Research Paradigm in Adversarial NLP](https://mlsanity.com/paper/2210.10683/) (2022)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Understanding and Detecting Hallucinations in Neural Machine Translation via Model Introspection](https://mlsanity.com/paper/2301.07779/) (2023)
- [Faithfulness in Natural Language Generation: A Systematic Survey of Analysis, Evaluation and Optimization Methods](https://mlsanity.com/paper/2203.05227/) (2022)
- [Improving Zero-shot Cross-lingual Transfer between Closely Related Languages by injecting Character-level Noise](https://mlsanity.com/paper/2109.06772/) (2021)
- [To Augment or Not to Augment? A Comparative Study on Text Augmentation Techniques for Low-Resource NLP](https://mlsanity.com/paper/2111.09618/) (2021)
- [On the Robustness of Intent Classification and Slot Labeling in Goal-oriented Dialog Systems to Real-world Noise](https://mlsanity.com/paper/2104.07149/) (2021)
- [CipherDAug: Ciphertext based Data Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2204.00665/) (2022)
- [Robustification of Multilingual Language Models to Real-world Noise in Crosslingual Zero-shot Settings with Robust Contrastive Pretraining](https://mlsanity.com/paper/2210.04782/) (2022)
- [Did Translation Models Get More Robust Without Anyone Even Noticing?](https://mlsanity.com/paper/2403.03923/) (2024)
- [Empirical Error Modeling Improves Robustness of Noisy Neural Sequence Labeling](https://mlsanity.com/paper/2105.11872/) (2021)
- [Improving Black-box Robustness with In-Context Rewriting](https://mlsanity.com/paper/2402.08225/) (2024)
- [Addressing the Vulnerability of NMT in Input Perturbations](https://mlsanity.com/paper/2104.09810/) (2021)
- [BERTwich: Extending BERT's Capabilities to Model Dialectal and Noisy Text](https://mlsanity.com/paper/2311.00116/) (2023)
- [Synthetic Data Generation and Joint Learning for Robust Code-Mixed Translation](https://mlsanity.com/paper/2403.16771/) (2024)
- [Stochastic Monkeys at Play: Random Augmentations Cheaply Break LLM Safety Alignment](https://mlsanity.com/paper/2411.02785/) (2024)
- [Generating Authentic Adversarial Examples beyond Meaning-preserving with Doubly Round-trip Translation](https://mlsanity.com/paper/2204.08689/) (2022)
- [Improving the Robustness of Summarization Models by Detecting and Removing Input Noise](https://mlsanity.com/paper/2212.09928/) (2022)
- [Robust Learning for Text Classification with Multi-source Noise Simulation and Hard Example Mining](https://mlsanity.com/paper/2107.07113/) (2021)
- [From Adversarial Arms Race to Model-centric Evaluation: Motivating a Unified Automatic Robustness Evaluation Framework](https://mlsanity.com/paper/2305.18503/) (2023)
- [ViPE: Visualise Pretty-much Everything](https://mlsanity.com/paper/2310.10543/) (2023)
- [Ask Language Model to Clean Your Noisy Translation Data](https://mlsanity.com/paper/2310.13469/) (2023)
- [Semantics or spelling? Probing contextual word embeddings with orthographic noise](https://mlsanity.com/paper/2408.04162/) (2024)
- [Misspellings in Natural Language Processing: A survey](https://mlsanity.com/paper/2501.16836/) (2025)
- [Sarang at DEFACTIFY 4.0: Detecting AI-Generated Text Using Noised Data and an Ensemble of DeBERTa Models](https://mlsanity.com/paper/2502.16857/) (2025)
- [Visual Cues and Error Correction for Translation Robustness](https://mlsanity.com/paper/2103.07352/) (2021)
- [Secoco: Self-Correcting Encoding for Neural Machine Translation](https://mlsanity.com/paper/2108.12137/) (2021)
- [PAEG: Phrase-level Adversarial Example Generation for Neural Machine Translation](https://mlsanity.com/paper/2201.02009/) (2022)

## Abstract (arXiv)

We consider the problem of making machine translation more robust to character-level variation at the source side, such as typos. Existing methods achieve greater coverage by applying subword models such as byte-pair encoding (BPE) and character-level encoders, but these methods are highly sensitive to spelling mistakes. We show how training on a mild amount of random synthetic noise can dramatically improve robustness to these variations, without diminishing performance on clean text. We focus on translation performance on natural noise, as captured by frequent corrections in Wikipedia edit logs, and show that robustness to such noise can be achieved using a balanced diet of simple synthetic noises at training time, without access to the natural noise data or distribution.