mlsanityData to 7 Oct 2026

Test Distribution-Aware Active Learning: A Principled Approach Against Distribution Shift and Outliers

arXiv 2106.11719 ↗First posted 22 Jun 2021cs.LG stat.MLcited by 22

Not on the map. Why?

Abstract (arXiv)

Expanding on MacKay (1992), we argue that conventional model-based methods for active learning - like BALD - have a fundamental shortfall: they fail to directly account for the test-time distribution of the input variables. This can lead to pathologies in the acquisition strategy, as what is maximally informative for model parameters may not be maximally informative for prediction: for example, when the data in the pool set is more dispersed than that of the final prediction task, or when the distribution of pool and test samples differs. To correct this, we revisit an acquisition strategy that is based on maximizing the expected information gained about possible future predictions, referring to this as the Expected Predictive Information Gain (EPIG). As EPIG does not scale well for batch acquisition, we further examine an alternative strategy, a hybrid between BALD and EPIG, which we call the Joint Expected Predictive Information Gain (JEPIG). We consider using both for active learning with Bayesian neural networks on a variety of datasets, examining the behavior under distribution shift in the pool set.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 16 of its 43 references that are in the map. First posted on arXiv 22 Jun 2021.

Builds on

16 of 43 references.

References · 16

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  5. 2018Deep Anomaly Detection with Outlier Exposurecited by 939
  6. 2017Deep Bayesian Active Learning with Image Datacited by 812
  7. 2019Deep Batch Active Learning by Diverse, Uncertain Gradient Lower Boundscited by 602
  8. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
  9. 2019BatchBALD: Efficient and Diverse Batch Acquisition for Deep Bayesian Active Learningcited by 428
  10. 2018Understanding Measures of Uncertainty for Adversarial Example Detectioncited by 177
  11. 2019Variational Bayesian Optimal Experimental Designcited by 116
  12. 2021Gone Fishing: Neural Active Learning with Fisher Embeddingscited by 88
4 more
  1. 2017On Nesting Monte Carlo Estimatorscited by 82
  2. 2021On Statistical Bias In Active Learning: How and When To Fix Itcited by 72
  3. 2020Predictive Information Accelerates Learning in RLcited by 56
  4. 2020Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?cited by 21

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 3; 2023: 9; 2024: 2; 2025: 4; 2026: 3.

22 papers.

Sort:

  1. 2023Anchor Points: Benchmarking Models with Much Fewer Examplescited by 84
  2. 2023Active Learning Principles for In-Context Learning with Large Language Modelscited by 53
  3. 2022Unifying Approaches in Active Learning and Active Sampling via Fisher Information and Information-Theoretic Quantitiescited by 28
  4. 2023DiscoBAX: Discovery of Optimal Intervention Sets in Genomic Experiment Designcited by 12
  5. 2024Making Better Use of Unlabelled Data in Bayesian Active Learningcited by 10
  6. 2023On the Limitations of Simulating Active Learningcited by 9
  7. 2022Active Learning with Expected Error Reductioncited by 7
  8. 2024Advancing Deep Active Learning & Data Subset Selection: Unifying Principles with Information-Theory Intuitionscited by 7
  9. 2023Investigating Multi-source Active Learning for Natural Language Inferencecited by 6
  10. 2023Training-Free Neural Active Learning with Initialization-Robustness Guaranteescited by 5
  11. 2022Marginal and Joint Cross-Entropies & Predictives for Online Bayesian Inference, Active Learning, and Active Samplingcited by 3
  12. 2023MoBYv2AL: Self-supervised Active Learning for Image Classificationcited by 3
10 more
  1. 2023ASPEST: Bridging the Gap Between Active Learning and Selective Predictioncited by 3
  2. 2025Distributionally Robust Active Learning for Gaussian Process Regressioncited by 3
  3. 2021Scalable Batch-Mode Deep Bayesian Active Learning via Equivalence Class Annealingcited by 2
  4. 2025Informed Initialization for Bayesian Optimization and Active Learningcited by 2
  5. 2023Learning Objective-Specific Active Learning Strategies with Attentive Neural Processescited by 1
  6. 2025Batch Bayesian Active Learning with Partial Batch Label Samplingcited by 0
  7. 2025Beyond Binary Out-of-Distribution Detection: Characterizing Distributional Shifts with Multi-Statistic Diffusion Trajectoriescited by 0
  8. 2026CAAL: Confidence-Aware Active Learning for Heteroscedastic Atmospheric Regressioncited by 0
  9. 2026Active Learning for Gaussian Process Regression Under Self-Induced Boltzmann Weightscited by 0
  10. 2026NodeImport: Imbalanced Node Classification with Node Importance Assessmentcited by 0