# Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy

arXiv: https://arxiv.org/abs/2607.18984 · HTML: https://mlsanity.com/paper/2607.18984/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 50 of its 110 references that are in the map.
- First posted on arXiv 21 Jul 2026.

## Builds on (most cited first, 30 of 50)

- [On The Power of Curriculum Learning in Training Deep Networks](https://mlsanity.com/paper/1904.03626/) (2019)
- [Competence-based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/1903.09848/) (2019)
- [Curriculum Learning and Minibatch Bucketing in Neural Machine Translation](https://mlsanity.com/paper/1707.09533/) (2017)
- [Norm-Based Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2006.02014/) (2020)
- [Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/1905.05816/) (2019)
- [Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives](https://mlsanity.com/paper/1905.10847/) (2019)
- [Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networks](https://mlsanity.com/paper/1611.06204/) (2016)
- [CDL: Curriculum Dual Learning for Emotion-Controllable Response Generation](https://mlsanity.com/paper/2005.00329/) (2020)
- [Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferences](https://mlsanity.com/paper/2403.07230/) (2024)
- [Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning](https://mlsanity.com/paper/2103.07552/) (2021)
- [Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning](https://mlsanity.com/paper/2102.03554/) (2021)
- [Code Execution with Pre-trained Language Models](https://mlsanity.com/paper/2305.05383/) (2023)
- [Reinforcement Learning based Curriculum Optimization for Neural Machine Translation](https://mlsanity.com/paper/1903.00041/) (2019)
- [Self-Paced Learning for Neural Machine Translation](https://mlsanity.com/paper/2010.04505/) (2020)
- [Dialogue Response Selection with Hierarchical Curriculum Learning](https://mlsanity.com/paper/2012.14756/) (2020)
- [Dynamic Data Selection and Weighting for Iterative Back-Translation](https://mlsanity.com/paper/2004.03672/) (2020)
- [Extracting Symptoms and their Status from Clinical Conversations](https://mlsanity.com/paper/1906.02239/) (2019)
- [Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning](https://mlsanity.com/paper/1605.03852/) (2016)
- [Dynamic Data Selection for Curriculum Learning via Ability Estimation](https://mlsanity.com/paper/2011.00080/) (2020)
- [Dynamically Composing Domain-Data Selection with Clean-Data Selection by "Co-Curricular Learning" for Neural Machine Translation](https://mlsanity.com/paper/1906.01130/) (2019)
- [An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models](https://mlsanity.com/paper/2203.09486/) (2022)
- [Learning Like Humans: Advancing LLM Reasoning Capabilities via Adaptive Difficulty Curriculum Learning and Expert-Guided Self-Reformulation](https://mlsanity.com/paper/2505.08364/) (2025)
- [Learning with Noise: Enhance Distantly Supervised Relation Extraction with Dynamic Transition Matrix](https://mlsanity.com/paper/1705.03995/) (2017)
- [PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation](https://mlsanity.com/paper/2208.08110/) (2022)
- [Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking](https://mlsanity.com/paper/2212.07617/) (2022)
- [Step-by-Step Mastery: Enhancing Soft Constraint Following Ability of Large Language Models](https://mlsanity.com/paper/2501.04945/) (2025)
- [Dynamic Curriculum Learning for Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2011.14608/) (2020)
- [Language models can learn implicit multi-hop reasoning, but only if they have lots of training data](https://mlsanity.com/paper/2505.17923/) (2025)
- [Hierarchical Curriculum Learning for AMR Parsing](https://mlsanity.com/paper/2110.07855/) (2021)
- [Generic and Trend-aware Curriculum Learning for Relation Extraction in Graph Neural Networks](https://mlsanity.com/paper/2205.08625/) (2022)

## Abstract (arXiv)

Despite more than a decade of curriculum learning (CL) research in NLP, the field lacks a principled account of which difficulty function or scheduler to use for a given problem. To understand what has hindered progress towards this account, we propose a fine-grained taxonomy separating difficulty evaluation from training scheduling to enable systematic analysis of CL strategies. For difficulty evaluation, we distinguish attribution source and task dependence, revealing difficulty as a perspectival concept encoding different assumptions about what makes an instance hard to learn. For scheduling, we provide the first formalisation of CL schedulers in terms of expected training contribution, enabling comparison across implementations by introducing retention regimes and monotonicity properties. Applied in a dedicated analysis of CL works in NLP, our taxonomy reveals a systematic incomparability problem: prior works conflate distinct notions of difficulty and scheduling, often pursuing different objectives under the same CL label -- hindering comparison and the accumulation of a coherent evidence base. Beyond diagnosis, the taxonomy supports the design, analysis, and comparison of CL strategies, and motivates evaluation practices that disentangle the sources of observed improvement.