mlsanityData to 7 Oct 2026

Epi-Curriculum: Episodic Curriculum Learning for Low-Resource Domain Adaptation in Neural Machine Translation

arXiv 2309.02640 ↗First posted 6 Sep 2023cs.LG cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) models have become successful, but their performance remains poor when translating on new domains with a limited number of data. In this paper, we present a novel approach Epi-Curriculum to address low-resource domain adaptation (DA), which contains a new episodic training framework along with denoised curriculum learning. Our episodic training framework enhances the model's robustness to domain shift by episodically exposing the encoder/decoder to an inexperienced decoder/encoder. The denoised curriculum learning filters the noised data and further improves the model's adaptability by gradually guiding the learning process from easy to more difficult tasks. Experiments on English-German and English-Romanian translation show that: (i) Epi-Curriculum improves both model's robustness and adaptability in seen and unseen domains; (ii) Our episodic training framework enhances the encoder and decoder's robustness to domain shift.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 38 of its 58 references that are in the map. First posted on arXiv 6 Sep 2023.

Builds on

38 of 58 references.

References · 38

  1. 2017Attention Is All You Needcited by 59,932
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  3. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,874
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  9. 2018On First-Order Meta-Learning Algorithmscited by 1,003
  10. 2018Adafactor: Adaptive Learning Rates with Sublinear Memory Costcited by 896
  11. 2017Deep Mutual Learningcited by 621
  12. 2017Six Challenges for Neural Machine Translationcited by 334
26 more
  1. 2019Episodic Training for Domain Generalizationcited by 245
  2. 2019Simple, Scalable Adaptation for Neural Machine Translationcited by 239
  3. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  4. 2022Generalized Federated Learning via Sharpness Aware Minimizationcited by 144
  5. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  6. 2018Meta-Learning for Low-Resource Neural Machine Translationcited by 105
  7. 2018A Survey of Domain Adaptation for Neural Machine Translationcited by 64
  8. 2019Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 59
  9. 2018An Empirical Exploration of Curriculum Learning for Neural Machine Translationcited by 58
  10. 2018Denoising Neural Machine Translation Training with Trusted Data and Online Data Selectioncited by 43
  11. 2019Reinforcement Learning based Curriculum Optimization for Neural Machine Translationcited by 29
  12. 2021Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 28
  13. 2019Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translationcited by 20
  14. 2020Meta-Learning for Few-Shot NMT Adaptationcited by 20
  15. 2018Compact Personalized Models for Neural Machine Translationcited by 19
  16. 2019MetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translationcited by 17
  17. 2020Investigating Catastrophic Forgetting During Continual Training for Neural Machine Translationcited by 17
  18. 2019Learning a Multi-Domain Curriculum for Neural Machine Translationcited by 16
  19. 2021The Trade-offs of Domain Adaptation for Neural Language Modelscited by 14
  20. 2021Improving Both Domain Robustness and Domain Adaptability in Machine Translationcited by 11
  21. 2021Towards User-Driven Neural Machine Translationcited by 10
  22. 2020Unsupervised Neural Machine Translation for Low-Resource Domains via Meta-Learningcited by 9
  23. 2020Addressing Exposure Bias With Document Minimum Risk Training: Cambridge at the WMT20 Biomedical Translation Taskcited by 8
  24. 2020Multi-Adversarial Learning for Cross-Lingual Word Embeddingscited by 7
  25. 2016Boosting Neural Machine Translationcited by 4
  26. 2021Improving Graph-based Sentence Ordering with Iteratively Predicted Pairwise Orderingscited by 4

Built on by

Not cited yet.