mlsanityData to 7 Oct 2026

Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles

arXiv 1612.01474 ↗First posted 5 Dec 2016stat.ML cs.LGcited by 3,888

Not on the map. Why?

Cited by 66 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep neural networks (NNs) are powerful black box predictors that have recently achieved impressive performance on a wide spectrum of tasks. Quantifying predictive uncertainty in NNs is a challenging and yet unsolved problem. Bayesian NNs, which learn a distribution over weights, are currently the state-of-the-art for estimating predictive uncertainty; however these require significant modifications to the training procedure and are computationally expensive compared to standard (non-Bayesian) NNs. We propose an alternative to Bayesian NNs that is simple to implement, readily parallelizable, requires very little hyperparameter tuning, and yields high quality predictive uncertainty estimates. Through a series of experiments on classification and regression benchmarks, we demonstrate that our method produces well-calibrated uncertainty estimates which are as good or better than approximate Bayesian NNs. To assess robustness to dataset shift, we evaluate the predictive uncertainty on test examples from known and unknown distributions, and show that our method is able to express higher uncertainty on out-of-distribution examples. We demonstrate the scalability of our method by evaluating predictive uncertainty estimates on ImageNet.

Key facts. Cited by 3,888 papers in the map (data through 7 Oct 2026). First posted on arXiv 5 Dec 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 3; 2019: 1; 2020: 3; 2021: 555; 2022: 570; 2023: 635; 2024: 614; 2025: 722; 2026: 785.

3,888 papers.

Sort:

By area:

  1. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,769
  2. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  3. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  4. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,366
  5. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
  6. 2021Generalized Out-of-Distribution Detection: A Surveycited by 796
  7. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  8. 2021Robust fine-tuning of zero-shot modelscited by 785
  9. 2021A Survey of Uncertainty in Deep Neural Networkscited by 675
  10. 2020MOPO: Model-based Offline Policy Optimizationcited by 630
  11. 2022Out-of-Distribution Detection with Deep Nearest Neighborscited by 596
  12. 2021ReAct: Out-of-distribution Detection With Rectified Activationscited by 486
38 more
  1. 2018Scalable agent alignment via reward modeling: a research directioncited by 438
  2. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  3. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 359
  4. 2021On the Importance of Gradients for Detecting Distributional Shifts in the Wildcited by 338
  5. 2021A Field Guide to Federated Optimizationcited by 312
  6. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  7. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  8. 2022VOS: Learning What You Don't Know by Virtual Outlier Synthesiscited by 300
  9. 2021Unsolved Problems in ML Safetycited by 291
  10. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 286
  11. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  12. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 267
  13. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  14. 2022Trajectory-guided Control Prediction for End-to-end Autonomous Driving: A Simple yet Strong Baselinecited by 247
  15. 2018HG-DAgger: Interactive Imitation Learning with Human Expertscited by 245
  16. 2021MOS: Towards Scaling Out-of-distribution Detection for Large Semantic Spacecited by 233
  17. 2023Rewarded soups: towards Pareto-optimal alignment by interpolating weights fine-tuned on diverse rewardscited by 232
  18. 2023Reward Model Ensembles Help Mitigate Overoptimizationcited by 225
  19. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  20. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  21. 2023PDE-Refiner: Achieving Accurate Long Rollouts with Neural PDE Solverscited by 205
  22. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  23. 2023Helping or Herding? Reward Model Ensembles Mitigate but do not Eliminate Reward Hackingcited by 195
  24. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  25. 2022Datamodels: Predicting Predictions from Training Datacited by 186
  26. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  27. 2022Biological Sequence Design with GFlowNetscited by 172
  28. 2022Trusted Multi-View Classification with Dynamic Evidential Fusioncited by 167
  29. 2021DICE: Leveraging Sparsification for Out-of-Distribution Detectioncited by 166
  30. 2024Fact-Checking the Output of Large Language Models via Token-Level Uncertainty Quantificationcited by 159
  31. 2024Kernel Language Entropy: Fine-grained Uncertainty Quantification for LLMs from Semantic Similaritiescited by 156
  32. 2021Well-tuned Simple Nets Excel on Tabular Datasetscited by 151
  33. 2021SegmentMeIfYouCan: A Benchmark for Anomaly Segmentationcited by 150
  34. 2022Diverse Weight Averaging for Out-of-Distribution Generalizationcited by 148
  35. 2023Shifting Attention to Relevance: Towards the Predictive Uncertainty Quantification of Free-Form Large Language Modelscited by 147
  36. 2022LIFT: Language-Interfaced Fine-Tuning for Non-Language Machine Learning Taskscited by 146
  37. 2022SHIFT: A Synthetic Driving Dataset for Continuous Multi-Task Domain Adaptationcited by 146
  38. 2022On Feature Learning in the Presence of Spurious Correlationscited by 145

50 most cited of 3,888 All 3,888 (JSON)