# Can Synthetic Translations Improve Bitext Quality?

arXiv: https://arxiv.org/abs/2203.07643 · HTML: https://mlsanity.com/paper/2203.07643/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 52 references that are in the map.
- First posted on arXiv 15 Mar 2022.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [mT5: A massively multilingual pre-trained text-to-text transformer](https://mlsanity.com/paper/2010.11934/) (2020)
- [XNLI: Evaluating Cross-lingual Sentence Representations](https://mlsanity.com/paper/1809.05053/) (2018)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Non-Autoregressive Neural Machine Translation](https://mlsanity.com/paper/1711.02281/) (2017)
- [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://mlsanity.com/paper/1812.10464/) (2018)
- [PAWS-X: A Cross-lingual Adversarial Dataset for Paraphrase Identification](https://mlsanity.com/paper/1908.11828/) (2019)
- [Evaluating Gender Bias in Machine Translation](https://mlsanity.com/paper/1906.00591/) (2019)
- [Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets](https://mlsanity.com/paper/2103.12028/) (2021)
- [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://mlsanity.com/paper/1907.05791/) (2019)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?](https://mlsanity.com/paper/1804.06323/) (2018)
- [Understanding Knowledge Distillation in Non-autoregressive Machine Translation](https://mlsanity.com/paper/1911.02727/) (2019)
- [Modeling Coverage for Neural Machine Translation](https://mlsanity.com/paper/1601.04811/) (2016)
- [Reducing Gender Bias in Neural Machine Translation as a Domain Adaptation Problem](https://mlsanity.com/paper/2004.04498/) (2020)
- [Language ID in the Wild: Unexpected Challenges on the Path to a Thousand-Language Web Text Corpus](https://mlsanity.com/paper/2010.14571/) (2020)
- [On the Impact of Various Types of Noise on Neural Machine Translation](https://mlsanity.com/paper/1805.12282/) (2018)
- [Mitigating Gender Bias in Machine Translation with Target Gender Annotations](https://mlsanity.com/paper/2010.06203/) (2020)
- [Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corpora](https://mlsanity.com/paper/1809.00197/) (2018)
- [Back-Translation Sampling by Targeting Difficult Words in Neural Machine Translation](https://mlsanity.com/paper/1808.09006/) (2018)
- [Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment](https://mlsanity.com/paper/2101.00148/) (2021)
- [Controlling Neural Machine Translation Formality with Synthetic Supervision](https://mlsanity.com/paper/1911.08706/) (2019)
- [Detecting Fine-Grained Cross-Lingual Semantic Divergences without Supervision by Learning to Rank](https://mlsanity.com/paper/2010.03662/) (2020)
- [Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translation](https://mlsanity.com/paper/2010.02552/) (2020)
- [Bilingual Lexicon Induction through Unsupervised Machine Translation](https://mlsanity.com/paper/1907.10761/) (2019)

## Built on by (most cited first, 1 of 1)

- [Enhancing Biomedical Text Summarization and Question-Answering: On the Utility of Domain-Specific Pre-Training](https://mlsanity.com/paper/2307.04412/) (2023)

## Abstract (arXiv)

Synthetic translations have been used for a wide range of NLP tasks primarily as a means of data augmentation. This work explores, instead, how synthetic translations can be used to revise potentially imperfect reference translations in mined bitext. We find that synthetic samples can improve bitext quality without any additional bilingual supervision when they replace the originals based on a semantic equivalence classifier that helps mitigate NMT noise. The improved quality of the revised bitext is confirmed intrinsically via human evaluation and extrinsically through bilingual induction and MT tasks.