# Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learning

arXiv: https://arxiv.org/abs/1605.03852 · HTML: https://mlsanity.com/paper/1605.03852/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 May 2016.

## Built on by (most cited first, 22 of 22)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Few-Shot Text Classification with Triplet Networks, Data Augmentation, and Curriculum Learning](https://mlsanity.com/paper/2103.07552/) (2021)
- [Efficient Continual Pre-training for Building Domain Specific Large Language Models](https://mlsanity.com/paper/2311.08545/) (2023)
- [The Effect of Task Ordering in Continual Learning](https://mlsanity.com/paper/2205.13323/) (2022)
- [Identifying Untrustworthy Samples: Data Filtering for Open-domain Dialogues with Bayesian Optimization](https://mlsanity.com/paper/2109.06471/) (2021)
- [PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentation](https://mlsanity.com/paper/2208.08110/) (2022)
- [From Easy to Hard: Progressive Active Learning Framework for Infrared Small Target Detection with Single Point Supervision](https://mlsanity.com/paper/2412.11154/) (2024)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Effective Generative AI: The Human-Algorithm Centaur](https://mlsanity.com/paper/2406.10942/) (2024)
- [Digital Twin-based Anomaly Detection with Curriculum Learning in Cyber-physical Systems](https://mlsanity.com/paper/2309.15995/) (2023)
- [A Psychology-based Unified Dynamic Framework for Curriculum Learning](https://mlsanity.com/paper/2408.05326/) (2024)
- [Self-directed Machine Learning](https://mlsanity.com/paper/2201.01289/) (2022)
- [Using Paraphrases to Study Properties of Contextual Embeddings](https://mlsanity.com/paper/2207.05553/) (2022)
- [Understanding Complexity in VideoQA via Visual Program Generation](https://mlsanity.com/paper/2505.13429/) (2025)
- [Analyzing Curriculum Learning for Sentiment Analysis along Task Difficulty, Pacing and Visualization Axes](https://mlsanity.com/paper/2102.09990/) (2021)
- [Understanding the Training Speedup from Sampling with Approximate Losses](https://mlsanity.com/paper/2402.07052/) (2024)
- [Curriculum generation using Autoencoder based continuous optimization](https://mlsanity.com/paper/2106.08569/) (2021)
- [Divergence-Based Domain Transferability for Zero-Shot Classification](https://mlsanity.com/paper/2302.05735/) (2023)
- [CrowdTransfer: Enabling Crowd Knowledge Transfer in AIoT Community](https://mlsanity.com/paper/2407.06485/) (2024)
- [Prototype-Guided Curriculum Learning for Zero-Shot Learning](https://mlsanity.com/paper/2508.07771/) (2025)
- [Beyond Repetition: Text Simplification and Curriculum Learning for Data-Constrained Pretraining](https://mlsanity.com/paper/2509.24356/) (2025)
- [Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy](https://mlsanity.com/paper/2607.18984/) (2026)

## Abstract (arXiv)

We use Bayesian optimization to learn curricula for word representation learning, optimizing performance on downstream tasks that depend on the learned representations as features. The curricula are modeled by a linear ranking function which is the scalar product of a learned weight vector and an engineered feature vector that characterizes the different aspects of the complexity of each instance in the training corpus. We show that learning the curriculum improves performance on a variety of downstream tasks over random orders and in comparison to the natural corpus order.