# Deep Active Learning with Noise Stability

arXiv: https://arxiv.org/abs/2205.13340 · HTML: https://mlsanity.com/paper/2205.13340/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 78 references that are in the map.
- First posted on arXiv 26 May 2022.

## Builds on (most cited first, 30 of 35)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learning](https://mlsanity.com/paper/1906.08158/) (2019)
- [Learning Loss for Active Learning](https://mlsanity.com/paper/1905.03677/) (2019)
- [Stronger generalization bounds for deep nets via a compression approach](https://mlsanity.com/paper/1802.05296/) (2018)
- [Dilated Residual Networks](https://mlsanity.com/paper/1705.09914/) (2017)
- [Variational Adversarial Active Learning](https://mlsanity.com/paper/1904.00370/) (2019)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [Batch Active Learning at Scale](https://mlsanity.com/paper/2107.14263/) (2021)
- [A General Framework for Uncertainty Estimation in Deep Learning](https://mlsanity.com/paper/1907.06890/) (2019)
- [Scalable Marginal Likelihood Estimation for Model Selection in Deep Learning](https://mlsanity.com/paper/2104.04975/) (2021)
- [Learning Active Learning from Data](https://mlsanity.com/paper/1703.03365/) (2017)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [Bayesian Deep Learning via Subnetwork Inference](https://mlsanity.com/paper/2010.14689/) (2020)
- [Influence Selection for Active Learning](https://mlsanity.com/paper/2108.09331/) (2021)
- [Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Network](https://mlsanity.com/paper/1806.05473/) (2018)
- [Generative Adversarial Active Learning](https://mlsanity.com/paper/1702.07956/) (2017)
- [Sequential Graph Convolutional Network for Active Learning](https://mlsanity.com/paper/2006.10219/) (2020)
- [State-Relabeling Adversarial Active Learning](https://mlsanity.com/paper/2004.04943/) (2020)
- [Adversarial Sampling for Active Learning](https://mlsanity.com/paper/1808.06671/) (2018)
- [CEREALS - Cost-Effective REgion-based Active Learning for Semantic Segmentation](https://mlsanity.com/paper/1810.09726/) (2018)
- [Cost-Effective Active Learning for Melanoma Segmentation](https://mlsanity.com/paper/1711.09168/) (2017)
- [Effective Evaluation of Deep Active Learning on Image Classification Tasks](https://mlsanity.com/paper/2106.15324/) (2021)

## Built on by (most cited first, 14 of 14)

- [You Never Get a Second Chance To Make a Good First Impression: Seeding Active Learning for 3D Semantic Segmentation](https://mlsanity.com/paper/2304.11762/) (2023)
- [Deep Active Learning in the Presence of Label Noise: A Survey](https://mlsanity.com/paper/2302.11075/) (2023)
- [Next Generation Active Learning: Mixture of LLMs in the Loop](https://mlsanity.com/paper/2601.15773/) (2026)
- [Proactive Schemes: A Survey of Adversarial Attacks for Social Good](https://mlsanity.com/paper/2409.16491/) (2024)
- [LPLgrad: Optimizing Active Learning Through Gradient Norm Sample Selection and Auxiliary Model Training](https://mlsanity.com/paper/2411.15217/) (2024)
- [VIOLA: Towards Video In-Context Learning with Minimal Annotations](https://mlsanity.com/paper/2601.15549/) (2026)
- [ALScope: A Unified Toolkit for Deep Active Learning](https://mlsanity.com/paper/2508.04937/) (2025)
- [Salutary Labeling with Zero Human Annotation](https://mlsanity.com/paper/2405.17627/) (2024)
- [A Novel Active Learning Approach to Label One Million Unknown Malware Variants](https://mlsanity.com/paper/2507.02959/) (2025)
- [Combining Discrepancy-Confusion Uncertainty and Calibration Diversity for Active Fine-Grained Image Classification](https://mlsanity.com/paper/2509.24181/) (2025)
- [Unsupervised Active Learning via Natural Feature Progressive Framework](https://mlsanity.com/paper/2510.04939/) (2025)
- [Uncertainty-Aware Subset Selection for Robust Visual Explainability under Distribution Shifts](https://mlsanity.com/paper/2512.08445/) (2025)
- [FACT: A Simple and Efficient Framework for Active Finetuning](https://mlsanity.com/paper/2606.02079/) (2026)
- [CAENL: Collapse-Based Active Neural Learning with Objective-Aligned Representation Control](https://mlsanity.com/paper/2610.17509/) (2026)

## Abstract (arXiv)

Uncertainty estimation for unlabeled data is crucial to active learning. With a deep neural network employed as the backbone model, the data selection process is highly challenging due to the potential over-confidence of the model inference. Existing methods resort to special learning fashions (e.g. adversarial) or auxiliary models to address this challenge. This tends to result in complex and inefficient pipelines, which would render the methods impractical. In this work, we propose a novel algorithm that leverages noise stability to estimate data uncertainty. The key idea is to measure the output derivation from the original observation when the model parameters are randomly perturbed by noise. We provide theoretical analyses by leveraging the small Gaussian noise theory and demonstrate that our method favors a subset with large and diverse gradients. Our method is generally applicable in various tasks, including computer vision, natural language processing, and structural data analysis. It achieves competitive performance compared against state-of-the-art active learning baselines.