Depth Uncertainty Networks for Active Learning
Not on the map. Why?
Abstract (arXiv)
In active learning, the size and complexity of the training dataset changes over time. Simple models that are well specified by the amount of data available at the start of active learning might suffer from bias as more points are actively sampled. Flexible models that might be well suited to the full dataset can suffer from overfitting towards the start of active learning. We tackle this problem using Depth Uncertainty Networks (DUNs), a BNN variant in which the depth of the network, and thus its complexity, is inferred. We find that DUNs outperform other BNN variants on several active learning tasks. Importantly, we show that on the tasks in which DUNs perform best they present notably less overfitting than baselines.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 15 of its 22 references that are in the map. First posted on arXiv 13 Dec 2021.
Builds on
15 of 22 references.
References · 15
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
- 2013Visualizing and Understanding Convolutional Networkscited by 2,290
- 2016Identity Mappings in Deep Residual Networkscited by 1,965
- 2015Weight Uncertainty in Neural Networkscited by 554
- 2020Underspecification Presents Challenges for Credibility in Modern Machine Learningcited by 496
- 2015Variational Dropout and the Local Reparameterization Trickcited by 490
- 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
- 2019Subspace Inference for Bayesian Deep Learningcited by 95
- 2019'In-Between' Uncertainty in Bayesian Neural Networkscited by 80
- 2021On Statistical Bias In Active Learning: How and When To Fix Itcited by 75
- 2020Depth Uncertainty in Neural Networkscited by 57
3 more
- 2017Model Selection in Bayesian Neural Networks via Horseshoe Priorscited by 52
- 2021Stochastic Batch Acquisition: A Simple Baseline for Deep Active Learningcited by 36
- 2020Variational Depth Search in ResNetscited by 2
Built on by
Citing papers per year: 2024: 2.
2 papers.