# Secoco: Self-Correcting Encoding for Neural Machine Translation

arXiv: https://arxiv.org/abs/2108.12137 · HTML: https://mlsanity.com/paper/2108.12137/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Synthetic and Natural Noise Both Break Neural Machine Translation](https://mlsanity.com/paper/1711.02173/) (2017)
- [Levenshtein Transformer](https://mlsanity.com/paper/1905.11006/) (2019)
- [Generating Natural Adversarial Examples](https://mlsanity.com/paper/1710.11342/) (2017)
- [Robust Neural Machine Translation with Doubly Adversarial Inputs](https://mlsanity.com/paper/1906.02443/) (2019)
- [On Adversarial Examples for Character-Level Neural Machine Translation](https://mlsanity.com/paper/1806.09030/) (2018)
- [AdvAug: Robust Adversarial Augmentation for Neural Machine Translation](https://mlsanity.com/paper/2006.11834/) (2020)
- [On Evaluation of Adversarial Perturbations for Sequence-to-Sequence Models](https://mlsanity.com/paper/1903.06620/) (2019)
- [Training on Synthetic Noise Improves Robustness to Natural Noise in Machine Translation](https://mlsanity.com/paper/1902.01509/) (2019)
- [Denoising Neural Machine Translation Training with Trusted Data and Online Data Selection](https://mlsanity.com/paper/1809.00068/) (2018)
- [Effective Parallel Corpus Mining using Bilingual Sentence Embeddings](https://mlsanity.com/paper/1807.11906/) (2018)
- [Improving Robustness of Machine Translation with Synthetic Noise](https://mlsanity.com/paper/1902.09508/) (2019)
- [Dynamic Data Selection for Neural Machine Translation](https://mlsanity.com/paper/1708.00712/) (2017)
- [Filtering and Mining Parallel Data in a Joint Multilingual Space](https://mlsanity.com/paper/1805.09822/) (2018)
- [How Robust Are Character-Based Word Embeddings in Tagging and MT Against Wrod Scramlbing or Randdm Nouse?](https://mlsanity.com/paper/1704.04441/) (2017)
- [Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation](https://mlsanity.com/paper/1906.01130/) (2019)
- [Robust Neural Machine Translation with Joint Textual and Phonetic Embedding](https://mlsanity.com/paper/1810.06729/) (2018)
- [Target Conditioned Sampling: Optimizing Data Selection for Multilingual Neural Machine Translation](https://mlsanity.com/paper/1905.08212/) (2019)
- [Autocorrect in the Process of Translation -- Multi-task Learning Improves Dialogue Machine Translation](https://mlsanity.com/paper/2103.16189/) (2021)
- [Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation](https://mlsanity.com/paper/1805.00178/) (2018)
- [Exploring the Robustness of NMT Systems to Nonsensical Inputs](https://mlsanity.com/paper/1908.01165/) (2019)

## Built on by (most cited first, 2 of 2)

- [Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languages](https://mlsanity.com/paper/2308.16075/) (2023)
- [A Representation Level Analysis of NMT Model Robustness to Grammatical Errors](https://mlsanity.com/paper/2505.21224/) (2025)

## Abstract (arXiv)

This paper presents Self-correcting Encoding (Secoco), a framework that effectively deals with input noise for robust neural machine translation by introducing self-correcting predictors. Different from previous robust approaches, Secoco enables NMT to explicitly correct noisy inputs and delete specific errors simultaneously with the translation decoding process. Secoco is able to achieve significant improvements over strong baselines on two real-world test sets and a benchmark WMT dataset with good interpretability. We will make our code and dataset publicly available soon.