mlsanityData to 7 Oct 2026

Making Better Use of Unlabelled Data in Bayesian Active Learning

arXiv 2404.17249 ↗First posted 26 Apr 2024cs.LG stat.MLcited by 10

Not on the map. Why?

Abstract (arXiv)

Fully supervised models are predominant in Bayesian active learning. We argue that their neglect of the information present in unlabelled data harms not just predictive performance but also decisions about what data to acquire. Our proposed solution is a simple framework for semi-supervised Bayesian active learning. We find it produces better-performing models than either conventional Bayesian active learning or semi-supervised learning with randomly acquired data. It is also easier to scale up than the conventional approach. As well as supporting a shift towards semi-supervised models, our findings highlight the importance of studying models and acquisition methods in conjunction.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 84 of its 160 references that are in the map. First posted on arXiv 26 Apr 2024.

Builds on

84 of 160 references.

References · 84

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  7. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  8. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  9. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  10. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  11. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  12. 2017Billion-scale similarity search with GPUscited by 2,477
38 more
  1. 2020Improved Baselines with Momentum Contrastive Learningcited by 2,282
  2. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
  3. 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
  4. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  5. 2017Revisiting Unreasonable Effectiveness of Data in Deep Learning Eracited by 916
  6. 2017Deep Bayesian Active Learning with Image Datacited by 812
  7. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  8. 2018Disentangling by Factorisingcited by 614
  9. 2018Exploring the Limits of Weakly Supervised Pretrainingcited by 604
  10. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  11. 2015Variational Dropout and the Local Reparameterization Trickcited by 489
  12. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  13. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  14. 2018Understanding disentangling in β-VAEcited by 332
  15. 2022Masked Siamese Networks for Label-Efficient Learningcited by 301
  16. 2015Scalable Bayesian Optimization Using Deep Neural Networkscited by 284
  17. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  18. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  19. 2022Active Learning on a Budget: Opposite Strategies Suit High and Low Budgetscited by 131
  20. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  21. 2023Modern Bayesian Experimental Designcited by 121
  22. 2021Solo-learn: A Library of Self-supervised Methods for Visual Representation Learningcited by 116
  23. 2019Consistency-based Semi-supervised Active Learning: Towards Minimizing Labeling Costcited by 114
  24. 2021Priors in Bayesian Deep Learning: A Reviewcited by 101
  25. 2018Overfitting or perfect fitting? Risk bounds for classification and regression rules that interpolatecited by 99
  26. 2019Deep Active Learning: Unified and Principled Method for Query and Trainingcited by 92
  27. 2021A Study of Face Obfuscation in ImageNetcited by 83
  28. 2019Bayesian Batch Active Learning as Sparse Subset Approximationcited by 68
  29. 2022Do Bayesian Neural Networks Need To Be Fully Stochastic?cited by 63
  30. 2022Active Learning Through a Covering Lenscited by 62
  31. 2020All You Need is a Good Functional Prior for Bayesian Deep Learningcited by 61
  32. 2020State-Relabeling Adversarial Active Learningcited by 60
  33. 2023Prediction-Oriented Bayesian Active Learningcited by 60
  34. 2020ASGN: An Active Semi-supervised Graph Neural Network for Molecular Property Predictioncited by 58
  35. 2019Bayesian Generative Active Deep Learningcited by 57
  36. 2020A statistical theory of cold posteriors in deep neural networkscited by 53
  37. 2021PASS: An ImageNet replacement for self-supervised pretraining without humanscited by 51
  38. 2021Semi-Supervised Active Learning with Temporal Output Discrepancycited by 50

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 3; 2026: 5.

10 papers.

  1. 2024Rethinking Aleatoric and Epistemic Uncertaintycited by 40
  2. 2024Post-hoc Probabilistic Vision-Language Modelscited by 13
  3. 2025Optimizing Likelihoods via Mutual Information: Bridging Simulation-Based Inference and Bayesian Optimal Experimental Designcited by 2
  4. 2026A Federated Generalized Expectation-Maximization Algorithm for Mixture Models with an Unknown Number of Componentscited by 1
  5. 2026Loss-Driven Bayesian Active Learningcited by 1
  6. 2026Consistency Regularised Gradient Flows for Inverse Problemscited by 1
  7. 2025Prediction-Oriented Subsampling from Data Streamscited by 0
  8. 2025Active Learning with Task-Driven Representations for Messy Poolscited by 0
  9. 2026Active learning for photonic crystalscited by 0
  10. 2026FairBED: A Bayesian Experimental Design Approach to Gathering Fairer Datacited by 0