# Active Learning for Open-set Annotation

arXiv: https://arxiv.org/abs/2201.06758 · HTML: https://mlsanity.com/paper/2201.06758/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 34 references that are in the map.
- First posted on arXiv 18 Jan 2022.

## Builds on (most cited first, 7 of 7)

- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Towards Open Set Deep Networks](https://mlsanity.com/paper/1511.06233/) (2015)
- [Recent Advances in Open Set Recognition: A Survey](https://mlsanity.com/paper/1811.08581/) (2018)
- [Generative OpenMax for Multi-Class Open Set Classification](https://mlsanity.com/paper/1707.07418/) (2017)
- [Bayesian Generative Active Deep Learning](https://mlsanity.com/paper/1904.11643/) (2019)
- [Cost-Effective Training of Deep CNNs with Active Model Adaptation](https://mlsanity.com/paper/1802.05394/) (2018)
- [Improving Model Robustness by Adaptively Correcting Perturbation Levels with Active Queries](https://mlsanity.com/paper/2103.14824/) (2021)

## Built on by (most cited first, 30 of 34)

- [A Survey on Deep Active Learning: Recent Advances and New Frontiers](https://mlsanity.com/paper/2405.00334/) (2024)
- [A comprehensive survey on deep active learning in medical image analysis](https://mlsanity.com/paper/2310.14230/) (2023)
- [Active Generalized Category Discovery](https://mlsanity.com/paper/2403.04272/) (2024)
- [AIDE: An Automatic Data Engine for Object Detection in Autonomous Driving](https://mlsanity.com/paper/2403.17373/) (2024)
- [Entropic Open-set Active Learning](https://mlsanity.com/paper/2312.14126/) (2023)
- [PiCO: Peer Review in LLMs based on the Consistency Optimization](https://mlsanity.com/paper/2402.01830/) (2024)
- [Coreset Sampling from Open-Set for Fine-Grained Self-Supervised Learning](https://mlsanity.com/paper/2303.11101/) (2023)
- [LEGO-Learn: Label-Efficient Graph Open-Set Learning](https://mlsanity.com/paper/2410.16386/) (2024)
- [Bidirectional Uncertainty-Based Active Learning for Open Set Annotation](https://mlsanity.com/paper/2402.15198/) (2024)
- [A Unified Approach Towards Active Learning and Out-of-Distribution Detection](https://mlsanity.com/paper/2405.11337/) (2024)
- [Joint Out-of-Distribution Filtering and Data Discovery Active Learning](https://mlsanity.com/paper/2503.02491/) (2025)
- [OpenAL: An Efficient Deep Active Learning Framework for Open-Set Pathology Image Classification](https://mlsanity.com/paper/2307.05254/) (2023)
- [Inconsistency-Based Data-Centric Active Open-Set Annotation](https://mlsanity.com/paper/2401.04923/) (2024)
- [Deep Active Learning in the Open World](https://mlsanity.com/paper/2411.06353/) (2024)
- [Active Learning via Classifier Impact and Greedy Selection for Interactive Image Retrieval](https://mlsanity.com/paper/2412.02310/) (2024)
- [Rethinking Epistemic and Aleatoric Uncertainty for Active Open-Set Annotation: An Energy-Based Approach](https://mlsanity.com/paper/2502.19691/) (2025)
- [Rapid Image Labeling via Neuro-Symbolic Learning](https://mlsanity.com/paper/2306.10490/) (2023)
- [MUS-CDB: Mixed Uncertainty Sampling with Class Distribution Balancing for Active Annotation in Aerial Object Detection](https://mlsanity.com/paper/2212.02804/) (2022)
- [Deep Active Learning with Contrastive Learning Under Realistic Data Pool Assumptions](https://mlsanity.com/paper/2303.14433/) (2023)
- [Enhancing Cost Efficiency in Active Learning with Candidate Set Query](https://mlsanity.com/paper/2502.06209/) (2025)
- [Training Ensembles with Inliers and Outliers for Semi-supervised Active Learning](https://mlsanity.com/paper/2307.03741/) (2023)
- [How To Overcome Confirmation Bias in Semi-Supervised Image Classification By Active Learning](https://mlsanity.com/paper/2308.08224/) (2023)
- [Avoid Wasted Annotation Costs in Open-set Active Learning with Pre-trained Vision-Language Model](https://mlsanity.com/paper/2408.04917/) (2024)
- [OpenPath: Open-Set Active Learning for Pathology Image Classification via Pre-trained Vision-Language Models](https://mlsanity.com/paper/2506.15318/) (2025)
- [ALScope: A Unified Toolkit for Deep Active Learning](https://mlsanity.com/paper/2508.04937/) (2025)
- [Revisiting Unknowns: Towards Effective and Efficient Open-Set Active Learning](https://mlsanity.com/paper/2603.07898/) (2026)
- [Dirichlet-Based Coarse-to-Fine Example Selection For Open-Set Annotation](https://mlsanity.com/paper/2409.17607/) (2024)
- [ADAptation: Reconstruction-based Unsupervised Active Learning for Breast Ultrasound Diagnosis](https://mlsanity.com/paper/2507.00474/) (2025)
- [Unsupervised Active Learning via Natural Feature Progressive Framework](https://mlsanity.com/paper/2510.04939/) (2025)
- [SAMOSA: Sharpness Aware Minimization for Open Set Active learning](https://mlsanity.com/paper/2510.16757/) (2025)

## Abstract (arXiv)

Existing active learning studies typically work in the closed-set setting by assuming that all data examples to be labeled are drawn from known classes. However, in real annotation tasks, the unlabeled data usually contains a large amount of examples from unknown classes, resulting in the failure of most active learning methods. To tackle this open-set annotation (OSA) problem, we propose a new active learning framework called LfOSA, which boosts the classification performance with an effective sampling strategy to precisely detect examples from known classes for annotation. The LfOSA framework introduces an auxiliary network to model the per-example max activation value (MAV) distribution with a Gaussian Mixture Model, which can dynamically select the examples with highest probability from known classes in the unlabeled set. Moreover, by reducing the temperature T of the loss function, the detection model will be further optimized by exploiting both known and unknown supervision. The experimental results show that the proposed method can significantly improve the selection quality of known classes, and achieve higher classification accuracy with lower annotation cost than state-of-the-art active learning methods. To the best of our knowledge, this is the first work of active learning for open-set annotation.