mlsanityData to 14 Oct 2026

Bayesian Estimate of Mean Proper Scores for Diversity-Enhanced Active Learning

arXiv 2312.10116 ↗First posted 15 Dec 2023cs.LG cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

The effectiveness of active learning largely depends on the sampling efficiency of the acquisition function. Expected Loss Reduction (ELR) focuses on a Bayesian estimate of the reduction in classification error, and more general costs fit in the same framework. We propose Bayesian Estimate of Mean Proper Scores (BEMPS) to estimate the increase in strictly proper scores such as log probability or negative mean square error within this framework. We also prove convergence results for this general class of costs. To facilitate better experimentation with the new acquisition functions, we develop a complementary batch AL algorithm that encourages diversity in the vector of expected changes in scores for unlabeled data. To allow high-performance classifiers, we combine deep ensembles, and dynamic validation set construction on pretrained models, and further speed up the ensemble process with the idea of Monte Carlo Dropout. Extensive experiments on both texts and images show that the use of mean square error and log probability with BEMPS yields robust acquisition functions and well-calibrated classifiers, and consistently outperforms the others tested. The advantages of BEMPS over the others are further supported by a set of qualitative analyses, where we visualise their sampling behaviour using data maps and t-SNE plots.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 33 of its 71 references that are in the map. First posted on arXiv 15 Dec 2023.

Builds on

33 of 71 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  5. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  6. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  7. 2015Character-level Convolutional Networks for Text Classificationcited by 2,330
  8. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  9. 2017Deep Bayesian Active Learning with Image Datacited by 819
  10. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 610
  11. 2020A Survey of Deep Active Learningcited by 554
  12. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 432
21 more
  1. 2020Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamicscited by 425
  2. 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 384
  3. 2020Cold-start Active Learning through Self-supervised Language Modelingcited by 128
  4. 2017PubMed 200k RCT: a Dataset for Sequential Sentence Classification in Medical Abstractscited by 124
  5. 2018Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Studycited by 90
  6. 2019Diverse mini-batch Active Learningcited by 89
  7. 2020Masksembles for Uncertainty Estimationcited by 88
  8. 2021Gone Fishing: Neural Active Learning with Fisher Embeddingscited by 88
  9. 2015Simple, Robust and Optimal Ranking from Pairwise Comparisonscited by 70
  10. 2019Bayesian Batch Active Learning as Sparse Subset Approximationcited by 68
  11. 2021Mind Your Outliers! Investigating the Negative Impact of Outliers on Active Learning for Visual Question Answeringcited by 62
  12. 2020A Survey of Active Learning for Text Classification using Deep Neural Networkscited by 50
  13. 2017Active Learning Using Uncertainty Informationcited by 28
  14. 2021Diversity Enhanced Active Learning with Strictly Proper Scoring Rulescited by 22
  15. 2018Deep Ensemble Bayesian Active Learning : Addressing the Mode Collapse issue in Monte Carlo dropout via Ensemblescited by 20
  16. 2021Cost-Accuracy Aware Adaptive Labeling for Active Learningcited by 13
  17. 2016Active Deep Learning for Classification of Hyperspectral Imagescited by 11
  18. 2018Capsule Networks against Medical Imaging Data Challengescited by 9
  19. 2021Diversity-Aware Batch Active Learning for Dependency Parsingcited by 9
  20. 2020Investigating the Effectiveness of Representations Based on Pretrained Transformer-based Language Models in Active Learning for Labelling Text Datasetscited by 6
  21. 2017Efficient variational Bayesian neural network ensembles for outlier detectioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 2.

6 papers.

  1. 2024Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classificationcited by 6
  2. 2026Next Generation Active Learning: Mixture of LLMs in the Loopcited by 3
  3. 2026Loss-Driven Bayesian Active Learningcited by 2
  4. 2024Harnessing the Power of Beta Scoring in Deep Active Learning for Multi-Label Text Classificationcited by 1
  5. 2025ALScope: A Unified Toolkit for Deep Active Learningcited by 1
  6. 2024Multi-Label Bayesian Active Learning with Inter-Label Relationshipscited by 0