# PAL : Pretext-based Active Learning

arXiv: https://arxiv.org/abs/2010.15947 · HTML: https://mlsanity.com/paper/2010.15947/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2020.

## Built on by (most cited first, 9 of 9)

- [PT4AL: Using Self-Supervised Pretext Tasks for Active Learning](https://mlsanity.com/paper/2201.07459/) (2022)
- [Exploring Active 3D Object Detection from a Generalization Perspective](https://mlsanity.com/paper/2301.09249/) (2023)
- [KECOR: Kernel Coding Rate Maximization for Active 3D Object Detection](https://mlsanity.com/paper/2307.07942/) (2023)
- [Making Better Use of Unlabelled Data in Bayesian Active Learning](https://mlsanity.com/paper/2404.17249/) (2024)
- [BAL: Balancing Diversity and Novelty for Active Learning](https://mlsanity.com/paper/2312.15944/) (2023)
- [Downstream-Pretext Domain Knowledge Traceback for Active Learning](https://mlsanity.com/paper/2407.14720/) (2024)
- [Open-CRB: Towards Open World Active Learning for 3D Object Detection](https://mlsanity.com/paper/2310.10391/) (2023)
- [Active Learning with Task-Driven Representations for Messy Pools](https://mlsanity.com/paper/2510.25926/) (2025)
- [Are Candidate Models Really Needed for Active Learning?](https://mlsanity.com/paper/2605.14689/) (2026)

## Abstract (arXiv)

The goal of pool-based active learning is to judiciously select a fixed-sized subset of unlabeled samples from a pool to query an oracle for their labels, in order to maximize the accuracy of a supervised learner. However, the unsaid requirement that the oracle should always assign correct labels is unreasonable for most situations. We propose an active learning technique for deep neural networks that is more robust to mislabeling than the previously proposed techniques. Previous techniques rely on the task network itself to estimate the novelty of the unlabeled samples, but learning the task (generalization) and selecting samples (out-of-distribution detection) can be conflicting goals. We use a separate network to score the unlabeled samples for selection. The scoring network relies on self-supervision for modeling the distribution of the labeled samples to reduce the dependency on potentially noisy labels. To counter the paucity of data, we also deploy another head on the scoring network for regularization via multi-task learning and use an unusual self-balancing hybrid scoring function. Furthermore, we divide each query into sub-queries before labeling to ensure that the query has diverse samples. In addition to having a higher tolerance to mislabeling of samples by the oracle, the resultant technique also produces competitive accuracy in the absence of label noise. The technique also handles the introduction of new classes on-the-fly well by temporarily increasing the sampling rate of these classes.