mlsanityData to 7 Oct 2026

Is BatchEnsemble a Single Model? On Calibration and Diversity of Efficient Ensembles

arXiv 2601.16936 ↗First posted 23 Jan 2026cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

In resource-constrained and low-latency settings, uncertainty estimates must be efficiently obtained. Deep Ensembles provide robust epistemic uncertainty (EU) but require training multiple full-size models. BatchEnsemble aims to deliver ensemble-like EU at far lower parameter and memory cost by applying learned rank-1 perturbations to a shared base network. We show that BatchEnsemble not only underperforms Deep Ensembles but closely tracks a single model baseline in terms of accuracy, calibration and out-of-distribution (OOD) detection on CIFAR10/10C/SVHN. A controlled study on MNIST finds members are near-identical in function and parameter space, indicating limited capacity to realize distinct predictive modes. Thus, BatchEnsemble behaves more like a single model than a true ensemble.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 12 of its 19 references that are in the map. First posted on arXiv 23 Jan 2026.

Builds on

12 of 19 references.

References · 12

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  4. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  5. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,362
  6. 2016Concrete Problems in AI Safetycited by 1,582
  7. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  8. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  9. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 368
  10. 2019MNIST-C: A Robustness Benchmark for Computer Visioncited by 137
  11. 2022Packed-Ensembles for Efficient Uncertainty Estimationcited by 49
  12. 2020On Power Laws in Deep Ensemblescited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Evaluating Prediction Uncertainty Estimates from BatchEnsemblecited by 0
  2. 2026LoMETab: Beyond Rank-1 Ensembles for Tabular Deep Learningcited by 0
  3. 2026Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulationcited by 0
  4. 2026Uncertainty quantification for trustworthy deep learning: Methods and measurescited by 0