# Dynamic Data Selection for Neural Machine Translation

arXiv: https://arxiv.org/abs/1708.00712 · HTML: https://mlsanity.com/paper/1708.00712/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Aug 2017.

## Built on by (most cited first, 30 of 33)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Prevent the Language Model from being Overconfident in Neural Machine Translation](https://mlsanity.com/paper/2105.11098/) (2021)
- [AdaPrompt: Adaptive Model Training for Prompt-based NLP](https://mlsanity.com/paper/2202.04824/) (2022)
- [Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/2103.02262/) (2021)
- [CombLM: Adapting Black-Box Language Models through Small Fine-Tuned Models](https://mlsanity.com/paper/2305.16876/) (2023)
- [On the Domain Adaptation and Generalization of Pretrained Language Models: A Survey](https://mlsanity.com/paper/2211.03154/) (2022)
- [Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasets](https://mlsanity.com/paper/2302.13959/) (2023)
- [Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits](https://mlsanity.com/paper/2110.06997/) (2021)
- [The Trade-offs of Domain Adaptation for Neural Language Models](https://mlsanity.com/paper/2109.10274/) (2021)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Automatic Document Selection for Efficient Encoder Pretraining](https://mlsanity.com/paper/2210.10951/) (2022)
- [Adaptive Training Distributions with Scalable Online Bilevel Optimization](https://mlsanity.com/paper/2311.11973/) (2023)
- [On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation](https://mlsanity.com/paper/2109.07591/) (2021)
- [Improving Automated Program Repair with Domain Adaptation](https://mlsanity.com/paper/2212.11414/) (2022)
- [Critical Learning Periods: Leveraging Early Training Dynamics for Efficient Data Pruning](https://mlsanity.com/paper/2405.19462/) (2024)
- [Gradient-guided Loss Masking for Neural Machine Translation](https://mlsanity.com/paper/2102.13549/) (2021)
- [Quality and Quantity of Machine Translation References for Automatic Metrics](https://mlsanity.com/paper/2401.01283/) (2024)
- [Selecting Parallel In-domain Sentences for Neural Machine Translation Using Monolingual Texts](https://mlsanity.com/paper/2112.06096/) (2021)
- [Domain Adaptation of Machine Translation with Crowdworkers](https://mlsanity.com/paper/2210.15861/) (2022)
- [Secoco: Self-Correcting Encoding for Neural Machine Translation](https://mlsanity.com/paper/2108.12137/) (2021)
- [m4Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter](https://mlsanity.com/paper/2210.11912/) (2022)
- [Robust Domain Adaptation for Pre-trained Multilingual Neural Machine Translation Models](https://mlsanity.com/paper/2210.14979/) (2022)
- [SAGE: Sustainable Agent-Guided Expert-tuning for Culturally Attuned Translation in Low-Resource Southeast Asia](https://mlsanity.com/paper/2603.19931/) (2026)
- [Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in text](https://mlsanity.com/paper/2311.06440/) (2023)
- [Comprehensive Benchmarking of Entropy and Margin Based Scoring Metrics for Data Selection](https://mlsanity.com/paper/2311.16302/) (2023)
- [LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation](https://mlsanity.com/paper/2406.01441/) (2024)
- [Clustering Algorithms and RAG Enhancing Semi-Supervised Text Classification with Large LLMs](https://mlsanity.com/paper/2411.06175/) (2024)
- [Dynamic Jointly Batch Selection for Data Efficient Machine Translation Fine-Tuning](https://mlsanity.com/paper/2511.04406/) (2025)
- [Domain Curricula for Code-Switched MT at MixMT 2022](https://mlsanity.com/paper/2210.17463/) (2022)
- [Label-Free Multi-Domain Machine Translation with Stage-wise Training](https://mlsanity.com/paper/2305.03949/) (2023)

## Abstract (arXiv)

Intelligent selection of training data has proven a successful technique to simultaneously increase training efficiency and translation performance for phrase-based machine translation (PBMT). With the recent increase in popularity of neural machine translation (NMT), we explore in this paper to what extent and how NMT can also benefit from data selection. While state-of-the-art data selection (Axelrod et al., 2011) consistently performs well for PBMT, we show that gains are substantially lower for NMT. Next, we introduce dynamic data selection for NMT, a method in which we vary the selected subset of training data between different training epochs. Our experiments show that the best results are achieved when applying a technique we call gradual fine-tuning, with improvements up to +2.6 BLEU over the original data selection approach and up to +3.1 BLEU over a general baseline.