Pareto Optimization for Active Learning under Out-of-Distribution Data Scenarios
Not on the map. Why?
Abstract (arXiv)
Pool-based Active Learning (AL) has achieved great success in minimizing labeling cost by sequentially selecting informative unlabeled samples from a large unlabeled data pool and querying their labels from oracle/annotators. However, existing AL sampling strategies might not work well in out-of-distribution (OOD) data scenarios, where the unlabeled data pool contains some data samples that do not belong to the classes of the target task. Achieving good AL performance under OOD data scenarios is a challenging task due to the natural conflict between AL sampling strategies and OOD sample detection. AL selects data that are hard to be classified by the current basic classifier (e.g., samples whose predicted class probabilities have high entropy), while OOD samples tend to have more uniform predicted class probabilities (i.e., high entropy) than in-distribution (ID) data. In this paper, we propose a sampling scheme, Monte-Carlo Pareto Optimization for Active Learning (POAL), which selects optimal subsets of unlabeled samples with fixed batch size from the unlabeled data pool. We cast the AL sampling task as a multi-objective optimization problem, and thus we utilize Pareto optimization based on two conflicting objectives: (1) the normal AL data sampling scheme (e.g., maximum entropy), and (2) the confidence of not being an OOD sample. Experimental results show its effectiveness on both classical Machine Learning (ML) and Deep Learning (DL) tasks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 33 of its 87 references that are in the map. First posted on arXiv 4 Jul 2022.
Builds on
33 of 87 references.
References · 33
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2017Billion-scale similarity search with GPUscited by 2,477
- 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
- 2018A Simple Unified Framework for Detecting Out-of-Distribution Samples and Adversarial Attackscited by 1,472
- 2017Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networkscited by 1,325
- 2017Deep Bayesian Active Learning with Image Datacited by 812
- 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
- 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
- 2020A Survey of Deep Active Learningcited by 547
- 2019Likelihood Ratios for Out-of-Distribution Detectioncited by 451
21 more
- 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 351
- 2019Learning Loss for Active Learningcited by 335
- 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
- 2017Learning Active Learning from Datacited by 100
- 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 92
- 2020Submodular Combinatorial Information Measures with Applications in Machine Learningcited by 88
- 2022A Comparative Survey of Deep Active Learningcited by 88
- 2021SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarioscited by 83
- 2021On Statistical Bias In Active Learning: How and When To Fix Itcited by 72
- 2021Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answeringcited by 62
- 2020Scalable Active Learning for Object Detectioncited by 58
- 2021Open-set Label Noise Can Improve Robustness Against Inherent Label Noisecited by 58
- 2019Batch Active Learning Using Determinantal Point Processescited by 42
- 2021Low Budget Active Learning via Wasserstein Distance: An Integer Programming Approachcited by 34
- 2020Active Learning under Label Shiftcited by 23
- 2021Discrepancy-Based Active Learning for Domain Adaptationcited by 16
- 2021DeepAL: Deep Active Learning in Pythoncited by 13
- 2019ALiPy: Active Learning in Pythoncited by 11
- 2021Multiple-criteria Based Active Learning with Fixed-size Determinantal Point Processescited by 9
- 2018A novel active learning framework for classification: using weighted rank aggregation to achieve multiple query criteriacited by 5
- 2019Decision Programming for Multi-Stage Optimization under Uncertaintycited by 1
Built on by
Citing papers per year: 2024: 2; 2025: 0; 2026: 1.
3 papers.