# Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation

arXiv: https://arxiv.org/abs/1805.00178 · HTML: https://mlsanity.com/paper/1805.00178/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 May 2018.

## Built on by (most cited first, 6 of 6)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation](https://mlsanity.com/paper/2212.07571/) (2022)
- [TNG-CLIP:Training-Time Negation Data Generation for Negation Awareness of CLIP](https://mlsanity.com/paper/2505.18434/) (2025)
- [Secoco: Self-Correcting Encoding for Neural Machine Translation](https://mlsanity.com/paper/2108.12137/) (2021)
- [Curriculum optimization for low-resource speech recognition](https://mlsanity.com/paper/2202.08883/) (2022)
- [Separating the Wheat from the Chaff with BREAD: An open-source benchmark and metrics to detect redundancy in text](https://mlsanity.com/paper/2311.06440/) (2023)

## Abstract (arXiv)

Traditional Neural machine translation (NMT) involves a fixed training procedure where each sentence is sampled once during each epoch. In reality, some sentences are well-learned during the initial few epochs; however, using this approach, the well-learned sentences would continue to be trained along with those sentences that were not well learned for 10-30 epochs, which results in a wastage of time. Here, we propose an efficient method to dynamically sample the sentences in order to accelerate the NMT training. In this approach, a weight is assigned to each sentence based on the measured difference between the training costs of two iterations. Further, in each epoch, a certain percentage of sentences are dynamically sampled according to their weights. Empirical results based on the NIST Chinese-to-English and the WMT English-to-German tasks depict that the proposed method can significantly accelerate the NMT training and improve the NMT performance.