# Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation

arXiv: https://arxiv.org/abs/2103.02262 · HTML: https://mlsanity.com/paper/2103.02262/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 32 references that are in the map.
- First posted on arXiv 3 Mar 2021.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [On First-Order Meta-Learning Algorithms](https://mlsanity.com/paper/1803.02999/) (2018)
- [Six Challenges for Neural Machine Translation](https://mlsanity.com/paper/1706.03872/) (2017)
- [Competence-based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1903.09848/) (2019)
- [Meta-Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/1808.08437/) (2018)
- [A Survey of Domain Adaptation for Neural Machine Translation](https://mlsanity.com/paper/1806.00258/) (2018)
- [Norm-Based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2006.02014/) (2020)
- [Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/1905.05816/) (2019)
- [An Empirical Exploration of Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1811.00739/) (2018)
- [compare-mt: A Tool for Holistic Comparison of Language Generation Systems](https://mlsanity.com/paper/1903.07926/) (2019)
- [Dynamic Data Selection for Neural Machine Translation](https://mlsanity.com/paper/1708.00712/) (2017)
- [Meta-Learning for Few-Shot NMT Adaptation](https://mlsanity.com/paper/2004.02745/) (2020)
- [MetaMT,a MetaLearning Method Leveraging Multiple Domain Data for Low Resource Machine Translation](https://mlsanity.com/paper/1912.05467/) (2019)
- [Cynical Selection of Language Model Training Data](https://mlsanity.com/paper/1709.02279/) (2017)
- [Shared-Private Bilingual Word Embeddings for Neural Machine Translation](https://mlsanity.com/paper/1906.03100/) (2019)

## Built on by (most cited first, 28 of 28)

- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [A Review of Deep Learning for Video Captioning](https://mlsanity.com/paper/2304.11431/) (2023)
- [Progressive Multi-Granularity Training for Non-Autoregressive Translation](https://mlsanity.com/paper/2106.05546/) (2021)
- [Improving Neural Machine Translation by Bidirectional Training](https://mlsanity.com/paper/2109.07780/) (2021)
- [Boost Your Human Image Generation Model via Direct Preference Optimization](https://mlsanity.com/paper/2405.20216/) (2024)
- [On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translation](https://mlsanity.com/paper/2110.01811/) (2021)
- [Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking](https://mlsanity.com/paper/2212.07617/) (2022)
- [Improving Both Domain Robustness and Domain Adaptability in Machine Translation](https://mlsanity.com/paper/2112.08288/) (2021)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Large Language Model for Multi-Domain Translation: Benchmarking and Domain CoT Fine-tuning](https://mlsanity.com/paper/2410.02631/) (2024)
- [Self-Guided Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2105.04475/) (2021)
- [Difficulty-Aware Machine Translation Evaluation](https://mlsanity.com/paper/2107.14402/) (2021)
- [A Psychology-based Unified Dynamic Framework for Curriculum Learning](https://mlsanity.com/paper/2408.05326/) (2024)
- [MELT: Materials-aware Continued Pre-training for Language Model Adaptation to Materials Science](https://mlsanity.com/paper/2410.15126/) (2024)
- [ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2212.04262/) (2022)
- [Cold-start Bundle Recommendation via Popularity-based Coalescence and Curriculum Heating](https://mlsanity.com/paper/2310.03813/) (2023)
- [HOP to the Next Tasks and Domains for Continual Learning in NLP](https://mlsanity.com/paper/2402.18449/) (2024)
- [DaLC: Domain Adaptation Learning Curve Prediction for Neural Machine Translation](https://mlsanity.com/paper/2204.09259/) (2022)
- [m4Adapter: Multilingual Multi-Domain Adaptation for Machine Translation with a Meta-Adapter](https://mlsanity.com/paper/2210.11912/) (2022)
- [Curriculum Guided Domain Adaptation in the Dark](https://mlsanity.com/paper/2308.00956/) (2023)
- [BIOptimus: Pre-training an Optimal Biomedical Language Model with Curriculum Learning for Named Entity Recognition](https://mlsanity.com/paper/2308.08625/) (2023)
- [Clustering Pseudo Language Family in Multilingual Translation Models with Fisher Information Matrix](https://mlsanity.com/paper/2312.02820/) (2023)
- [A Two-Stage Prediction-Aware Contrastive Learning Framework for Multi-Intent NLU](https://mlsanity.com/paper/2405.02925/) (2024)
- [ERNetCL: A novel emotion recognition network in textual conversation based on curriculum learning strategy](https://mlsanity.com/paper/2308.06450/) (2023)
- [CLewR: Curriculum Learning with Restarts for Machine Translation Preference Learning](https://mlsanity.com/paper/2601.05858/) (2026)
- [Reinforcement Learning for Few-Shot Text Generation Adaptation](https://mlsanity.com/paper/2111.11030/) (2021)
- [Epi-Curriculum: Episodic Curriculum Learning for Low-Resource Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/2309.02640/) (2023)

## Abstract (arXiv)

Meta-learning has been sufficiently validated to be beneficial for low-resource neural machine translation (NMT). However, we find that meta-trained NMT fails to improve the translation performance of the domain unseen at the meta-training stage. In this paper, we aim to alleviate this issue by proposing a novel meta-curriculum learning for domain adaptation in NMT. During meta-training, the NMT first learns the similar curricula from each domain to avoid falling into a bad local optimum early, and finally learns the curricula of individualities to improve the model robustness for learning domain-specific knowledge. Experimental results on 10 different low-resource domains show that meta-curriculum learning can improve the translation performance of both familiar and unfamiliar domains. All the codes and data are freely available at https://github.com/NLP2CT/Meta-Curriculum.