# Meta-Learning Transferable Active Learning Policies by Deep Reinforcement Learning

arXiv: https://arxiv.org/abs/1806.04798 · HTML: https://mlsanity.com/paper/1806.04798/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jun 2018.

## Built on by (most cited first, 26 of 26)

- [How to Train Your MAML to Excel in Few-Shot Classification](https://mlsanity.com/paper/2106.16245/) (2021)
- [Unsupervised Representation Learning in Deep Reinforcement Learning: A Review](https://mlsanity.com/paper/2208.14226/) (2022)
- [DiscoBAX: Discovery of Optimal Intervention Sets in Genomic Experiment Design](https://mlsanity.com/paper/2312.04064/) (2023)
- [On Provably Robust Meta-Bayesian Optimization](https://mlsanity.com/paper/2206.06872/) (2022)
- [Bayesian Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2408.01694/) (2024)
- [A Survey on Active Deep Learning: From Model-driven to Data-driven](https://mlsanity.com/paper/2101.09933/) (2021)
- [ImitAL: Learning Active Learning Strategies from Synthetic Data](https://mlsanity.com/paper/2108.07670/) (2021)
- [Active Learning of Quantum System Hamiltonians yields Query Advantage](https://mlsanity.com/paper/2112.14553/) (2021)
- [Challenges and Opportunities for Machine Learning Classification of Behavior and Mental State from Images](https://mlsanity.com/paper/2201.11197/) (2022)
- [Deep reinforced active learning for multi-class image classification](https://mlsanity.com/paper/2206.13391/) (2022)
- [Active learning for interactive satellite image change detection](https://mlsanity.com/paper/2110.04250/) (2021)
- [Reinforcement-based frugal learning for satellite image change detection](https://mlsanity.com/paper/2203.11564/) (2022)
- [ImitAL: Learned Active Learning Strategy on Synthetic Data](https://mlsanity.com/paper/2208.11636/) (2022)
- [Learning Objective-Specific Active Learning Strategies with Attentive Neural Processes](https://mlsanity.com/paper/2309.05477/) (2023)
- [Reinforcement-based Display-size Selection for Frugal Satellite Image Change Detection](https://mlsanity.com/paper/2312.16965/) (2023)
- [Combining Bayesian Inference and Reinforcement Learning for Agent Decision Making: A Review](https://mlsanity.com/paper/2505.07911/) (2025)
- [Active Learning for Argument Mining: A Practical Approach](https://mlsanity.com/paper/2109.13611/) (2021)
- [Fair Active Learning: Solving the Labeling Problem in Insurance](https://mlsanity.com/paper/2112.09466/) (2021)
- [Frugal Learning of Virtual Exemplars for Label-Efficient Satellite Image Change Detection](https://mlsanity.com/paper/2203.11559/) (2022)
- [Frugal Reinforcement-based Active Learning](https://mlsanity.com/paper/2212.04868/) (2022)
- [Adversarial Virtual Exemplar Learning for Label-Frugal Satellite Image Change Detection](https://mlsanity.com/paper/2212.13974/) (2022)
- [Frugal Satellite Image Change Detection with Deep-Net Inversion](https://mlsanity.com/paper/2309.14781/) (2023)
- [Simulating, Fast and Slow: Learning Policies for Black-Box Optimization](https://mlsanity.com/paper/2406.04261/) (2024)
- [Label-frugal satellite image change detection with generative virtual exemplar learning](https://mlsanity.com/paper/2510.06926/) (2025)
- [Image augmentation with invertible networks in interactive satellite image change detection](https://mlsanity.com/paper/2510.18660/) (2025)
- [Cumulative Meta-Learning from Active Learning Queries for Robustness to Spurious Correlations](https://mlsanity.com/paper/2605.20771/) (2026)

## Abstract (arXiv)

Active learning (AL) aims to enable training high performance classifiers with low annotation cost by predicting which subset of unlabelled instances would be most beneficial to label. The importance of AL has motivated extensive research, proposing a wide variety of manually designed AL algorithms with diverse theoretical and intuitive motivations. In contrast to this body of research, we propose to treat active learning algorithm design as a meta-learning problem and learn the best criterion from data. We model an active learning algorithm as a deep neural network that inputs the base learner state and the unlabelled point set and predicts the best point to annotate next. Training this active query policy network with reinforcement learning, produces the best non-myopic policy for a given dataset. The key challenge in achieving a general solution to AL then becomes that of learner generalisation, particularly across heterogeneous datasets. We propose a multi-task dataset-embedding approach that allows dataset-agnostic active learners to be trained. Our evaluation shows that AL algorithms trained in this way can directly generalise across diverse problems.