# Learning to Sample: an Active Learning Framework

arXiv: https://arxiv.org/abs/1909.03585 · HTML: https://mlsanity.com/paper/1909.03585/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 11 of 11)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [Algorithm Selection for Deep Active Learning with Imbalanced Datasets](https://mlsanity.com/paper/2302.07317/) (2023)
- [Does Configuration Encoding Matter in Learning Software Performance? An Empirical Study on Encoding Schemes](https://mlsanity.com/paper/2203.15988/) (2022)
- [Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection](https://mlsanity.com/paper/2306.00765/) (2023)
- [Addressing practical challenges in Active Learning via a hybrid query strategy](https://mlsanity.com/paper/2110.03785/) (2021)
- [How To Overcome Confirmation Bias in Semi-Supervised Image Classification By Active Learning](https://mlsanity.com/paper/2308.08224/) (2023)
- [Importance of negative sampling in weak label learning](https://mlsanity.com/paper/2309.13227/) (2023)
- [Accountable Error Characterization](https://mlsanity.com/paper/2105.04707/) (2021)
- [Reinforced Meta Active Learning](https://mlsanity.com/paper/2203.04573/) (2022)
- [Hazards in Deep Learning Testing: Prevalence, Impact and Recommendations](https://mlsanity.com/paper/2309.05381/) (2023)
- [Towards Comparable Active Learning](https://mlsanity.com/paper/2311.18356/) (2023)

## Abstract (arXiv)

Meta-learning algorithms for active learning are emerging as a promising paradigm for learning the ``best'' active learning strategy. However, current learning-based active learning approaches still require sufficient training data so as to generalize meta-learning models for active learning. This is contrary to the nature of active learning which typically starts with a small number of labeled samples. The unavailability of large amounts of labeled samples for training meta-learning models would inevitably lead to poor performance (e.g., instabilities and overfitting). In our paper, we tackle these issues by proposing a novel learning-based active learning framework, called Learning To Sample (LTS). This framework has two key components: a sampling model and a boosting model, which can mutually learn from each other in iterations to improve the performance of each other. Within this framework, the sampling model incorporates uncertainty sampling and diversity sampling into a unified process for optimization, enabling us to actively select the most representative and informative samples based on an optimized integration of uncertainty and diversity. To evaluate the effectiveness of the LTS framework, we have conducted extensive experiments on three different classification tasks: image classification, salary level prediction, and entity resolution. The experimental results show that our LTS framework significantly outperforms all the baselines when the label budget is limited, especially for datasets with highly imbalanced classes. In addition to this, our LTS framework can effectively tackle the cold start problem occurring in many existing active learning approaches.