# Active Learning with Neural Networks: Insights from Nonparametric Statistics

arXiv: https://arxiv.org/abs/2210.08367 · HTML: https://mlsanity.com/paper/2210.08367/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 87 references that are in the map.
- First posted on arXiv 15 Oct 2022.

## Builds on (most cited first, 27 of 27)

- [Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Bounds](https://mlsanity.com/paper/1906.03671/) (2019)
- [Error bounds for approximations with deep ReLU networks](https://mlsanity.com/paper/1610.01145/) (2016)
- [A Survey of Deep Active Learning](https://mlsanity.com/paper/2009.00236/) (2020)
- [Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks](https://mlsanity.com/paper/1703.02930/) (2017)
- [Taming the Monster: A Fast and Simple Algorithm for Contextual Bandits](https://mlsanity.com/paper/1402.0555/) (2014)
- [Optimal approximation of continuous functions by very deep ReLU networks](https://mlsanity.com/paper/1802.03620/) (2018)
- [Deep Network Approximation for Smooth Functions](https://mlsanity.com/paper/2001.03040/) (2020)
- [Batch Active Learning at Scale](https://mlsanity.com/paper/2107.14263/) (2021)
- [A Function Space View of Bounded Norm Infinite Width ReLU Nets: The Multivariate Case](https://mlsanity.com/paper/1910.01635/) (2019)
- [Practical Contextual Bandits with Regression Oracles](https://mlsanity.com/paper/1803.01088/) (2018)
- [Gone Fishing: Neural Active Learning with Fisher Embeddings](https://mlsanity.com/paper/2106.09675/) (2021)
- [SIMILAR: Submodular Information Measures Based Active Learning In Realistic Scenarios](https://mlsanity.com/paper/2107.00717/) (2021)
- [Instance-Dependent Complexity of Contextual Bandits and Reinforcement Learning: A Disagreement-Based Perspective](https://mlsanity.com/paper/2010.03104/) (2020)
- [What Kinds of Functions do Deep Neural Networks Learn? Insights from Variational Spline Theory](https://mlsanity.com/paper/2105.03361/) (2021)
- [Fast convergence rates of deep neural networks for classification](https://mlsanity.com/paper/1812.03599/) (2018)
- [Active Learning for Cost-Sensitive Classification](https://mlsanity.com/paper/1703.01014/) (2017)
- [Near-Minimax Optimal Estimation With Shallow ReLU Neural Networks](https://mlsanity.com/paper/2109.08844/) (2021)
- [Active Learning at the ImageNet Scale](https://mlsanity.com/paper/2111.12880/) (2021)
- [Exponential Savings in Agnostic Active Learning through Abstention](https://mlsanity.com/paper/2102.00451/) (2021)
- [Neural Active Learning with Performance Guarantees](https://mlsanity.com/paper/2106.03243/) (2021)
- [Efficient Active Learning with Abstention](https://mlsanity.com/paper/2204.00043/) (2022)
- [Ridges, Neural Networks, and the Radon Transform](https://mlsanity.com/paper/2203.02543/) (2022)
- [Efficient and Parsimonious Agnostic Active Learning](https://mlsanity.com/paper/1506.08669/) (2015)
- [Adaptivity to Noise Parameters in Nonparametric Active Learning](https://mlsanity.com/paper/1703.05841/) (2017)
- [MaxiMin Active Learning in Overparameterized Model Classes}](https://mlsanity.com/paper/1905.12782/) (2019)
- [Nuances in Margin Conditions Determine Gains in Active Learning](https://mlsanity.com/paper/2110.08418/) (2021)
- [An Adaptive Strategy for Active Learning with Smooth Decision Boundary](https://mlsanity.com/paper/1711.09294/) (2017)

## Built on by (most cited first, 13 of 13)

- [Neural Active Learning Beyond Bandits](https://mlsanity.com/paper/2404.12522/) (2024)
- [Out-of-Distribution Learning with Human Feedback](https://mlsanity.com/paper/2408.07772/) (2024)
- [Test-Time Matching: Unlocking Compositional Reasoning in Multimodal Models](https://mlsanity.com/paper/2510.07632/) (2025)
- [Provably Neural Active Learning Succeeds via Prioritizing Perplexing Samples](https://mlsanity.com/paper/2406.03944/) (2024)
- [Bayesian Active Learning for Semantic Segmentation](https://mlsanity.com/paper/2408.01694/) (2024)
- [Active Testing of Large Language Models via Approximate Neyman Allocation](https://mlsanity.com/paper/2605.10075/) (2026)
- [Towards Multimodal Active Learning: Efficient Learning with Limited Paired Data](https://mlsanity.com/paper/2510.03247/) (2025)
- [Closing the Reflection Gap: A Free Calibration Bonus for Agentic RL](https://mlsanity.com/paper/2606.14211/) (2026)
- [Learning to Help in Multi-Class Settings](https://mlsanity.com/paper/2501.13810/) (2025)
- [Active Learning of Deep Neural Networks via Gradient-Free Cutting Planes](https://mlsanity.com/paper/2410.02145/) (2024)
- [Online Finetuning Decision Transformers with Pure RL Gradients](https://mlsanity.com/paper/2601.00167/) (2026)
- [A signal separation view of classification](https://mlsanity.com/paper/2509.24140/) (2025)
- [Active Learning with Foundation Model Priors: Efficient Learning under Class Imbalance](https://mlsanity.com/paper/2606.07630/) (2026)

## Abstract (arXiv)

Deep neural networks have great representation power, but typically require large numbers of training examples. This motivates deep active learning methods that can significantly reduce the amount of labeled training data. Empirical successes of deep active learning have been recently reported in the literature, however, rigorous label complexity guarantees of deep active learning have remained elusive. This constitutes a significant gap between theory and practice. This paper tackles this gap by providing the first near-optimal label complexity guarantees for deep active learning. The key insight is to study deep active learning from the nonparametric classification perspective. Under standard low noise conditions, we show that active learning with neural networks can provably achieve the minimax label complexity, up to disagreement coefficient and other logarithmic terms. When equipped with an abstention option, we further develop an efficient deep active learning algorithm that achieves mathsfpolylog(frac1ε) label complexity, without any low noise assumptions. We also provide extensions of our results beyond the commonly studied Sobolev/H"older spaces and develop label complexity guarantees for learning in Radon mathsfBV2 spaces, which have recently been proposed as natural function spaces associated with neural networks.