# Dynamic Data Selection and Weighting for Iterative Back-Translation

arXiv: https://arxiv.org/abs/2004.03672 · HTML: https://mlsanity.com/paper/2004.03672/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 25 of 25)

- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://mlsanity.com/paper/2402.07827/) (2024)
- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Task](https://mlsanity.com/paper/2111.02086/) (2021)
- [Data Augmentation for Sign Language Gloss Translation](https://mlsanity.com/paper/2105.07476/) (2021)
- [Meta Back-translation](https://mlsanity.com/paper/2102.07847/) (2021)
- [Don't Go Far Off: An Empirical Study on Neural Poetry Translation](https://mlsanity.com/paper/2109.02972/) (2021)
- [Confidence-Aware Scheduled Sampling for Neural Machine Translation](https://mlsanity.com/paper/2107.10427/) (2021)
- [On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation](https://mlsanity.com/paper/2109.07591/) (2021)
- [Comparison of Grammatical Error Correction Using Back-Translation Models](https://mlsanity.com/paper/2104.07848/) (2021)
- [Self-Guided Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2105.04475/) (2021)
- [Back Translation for Speech-to-text Translation Without Transcripts](https://mlsanity.com/paper/2305.08709/) (2023)
- [LLM See, LLM Do: Guiding Data Generation to Target Non-Differentiable Objectives](https://mlsanity.com/paper/2407.01490/) (2024)
- [Geographical Distance Is The New Hyperparameter: A Case Study Of Finding The Optimal Pre-trained Language For English-isiZulu Machine Translation](https://mlsanity.com/paper/2205.08621/) (2022)
- [Family of Origin and Family of Choice: Massively Parallel Lexiconized Iterative Pretraining for Severely Low Resource Machine Translation](https://mlsanity.com/paper/2104.05848/) (2021)
- [Improving Logical-Level Natural Language Generation with Topic-Conditioned Data Augmentation and Logical Form Generation](https://mlsanity.com/paper/2112.06240/) (2021)
- [Fix your Models by Fixing your Datasets](https://mlsanity.com/paper/2112.07844/) (2021)
- [DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation](https://mlsanity.com/paper/2204.09259/) (2022)
- [A Data-centric Framework for Improving Domain-specific Machine Reading Comprehension Datasets](https://mlsanity.com/paper/2304.00483/) (2023)
- [The Impact of Syntactic and Semantic Proximity on Machine Translation with Back-Translation](https://mlsanity.com/paper/2403.18031/) (2024)
- [Generalised Unsupervised Domain Adaptation of Neural Machine Translation with Cross-Lingual Data Selection](https://mlsanity.com/paper/2109.04292/) (2021)
- [Constructing Multilingual Code Search Dataset Using Neural Machine Translation](https://mlsanity.com/paper/2306.15604/) (2023)
- [From Priest to Doctor: Domain Adaptation for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2412.00966/) (2024)
- [Exploiting Language Relatedness in Machine Translation Through Domain Adaptation Techniques](https://mlsanity.com/paper/2303.01793/) (2023)
- [Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy](https://mlsanity.com/paper/2607.18984/) (2026)

## Abstract (arXiv)

Back-translation has proven to be an effective method to utilize monolingual data in neural machine translation (NMT), and iteratively conducting back-translation can further improve the model performance. Selecting which monolingual data to back-translate is crucial, as we require that the resulting synthetic data are of high quality and reflect the target domain. To achieve these two goals, data selection and weighting strategies have been proposed, with a common practice being to select samples close to the target domain but also dissimilar to the average general-domain text. In this paper, we provide insights into this commonly used approach and generalize it to a dynamic curriculum learning strategy, which is applied to iterative back-translation models. In addition, we propose weighting strategies based on both the current quality of the sentence and its improvement over the previous iteration. We evaluate our models on domain adaptation, low-resource, and high-resource MT settings and on two language pairs. Experimental results demonstrate that our methods achieve improvements of up to 1.8 BLEU points over competitive baselines.