mlsanityData to 14 Oct 2026

Packed-Ensembles for Efficient Uncertainty Estimation

arXiv 2210.09184 ↗First posted 17 Oct 2022cs.LG stat.MLcited by 49

Not on the map. Why?

Abstract (arXiv)

Deep Ensembles (DE) are a prominent approach for achieving excellent performance on key metrics such as accuracy, calibration, uncertainty estimation, and out-of-distribution detection. However, hardware limitations of real-world systems constrain to smaller ensembles and lower-capacity networks, significantly deteriorating their performance and properties. We introduce Packed-Ensembles (PE), a strategy to design and train lightweight structured ensembles by carefully modulating the dimension of their encoding space. We leverage grouped convolutions to parallelize the ensemble into a single shared backbone and forward pass to improve training and inference speeds. PE is designed to operate within the memory limits of a standard neural network. Our extensive research indicates that PE accurately preserves the properties of DE, such as diversity, and performs equally well in terms of accuracy, calibration, out-of-distribution detection, and robustness to distribution shift. We make our code available at https://github.com/ENSTA-U2IS/torch-uncertainty.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Builds on 57 of its 79 references that are in the map. First posted on arXiv 17 Oct 2022.

Builds on

57 of 79 references.

References · 57

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,475
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,183
  6. 2017On Calibration of Modern Neural Networkscited by 4,960
  7. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  8. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  9. 2016Wide Residual Networkscited by 3,070
  10. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,884
  11. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,877
  12. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
38 more
  1. 2016A Baseline for Detecting Misclassified and Out-of-Distribution Examples in Neural Networkscited by 2,385
  2. 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,352
  3. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  4. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,269
  5. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  6. 2019Searching for MobileNetV3cited by 1,994
  7. 2020The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalizationcited by 1,850
  8. 2019Natural Adversarial Examplescited by 1,408
  9. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  10. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
  11. 2016Group Equivariant Convolutional Networkscited by 1,165
  12. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  13. 2014Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Imagescited by 776
  14. 2018Loss Surfaces, Mode Connectivity, and Fast Ensembling of DNNscited by 588
  15. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  16. 2019A Simple Baseline for Bayesian Uncertainty in Deep Learningcited by 560
  17. 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
  18. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  19. 2022ViM: Out-Of-Distribution with Virtual-logit Matchingcited by 405
  20. 2020BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learningcited by 371
  21. 2021ResNet strikes back: An improved training procedure in timmcited by 368
  22. 2017Snapshot Ensembles: Train 1, get M for freecited by 350
  23. 2017Scalable Training of Artificial Neural Networks with Adaptive Sparse Connectivity inspired by Network Sciencecited by 328
  24. 2018Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problemcited by 323
  25. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  26. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  27. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 245
  28. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 210
  29. 2019Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Visioncited by 182
  30. 2020Training independent subnetworks for robust predictioncited by 177
  31. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  32. 2015Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inferencecited by 159
  33. 2017Towards Accurate Binary Convolutional Neural Networkcited by 149
  34. 2015Why M Heads are Better than One: Training a Diverse Ensemble of Deep Networkscited by 139
  35. 2020Masksembles for Uncertainty Estimationcited by 88
  36. 2020Can weight sharing outperform random architecture search? An investigation with TuNAScited by 68
  37. 2018A Modern Take on the Bias-Variance Tradeoff in Neural Networkscited by 65
  38. 2021SA-Net: Shuffle Attention for Deep Convolutional Neural Networkscited by 54

50 most cited of 57 All 57 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 12; 2025: 11; 2026: 16.

49 papers.

Sort:

By area:

  1. 2024TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensemblingcited by 113
  2. 2023A Survey on Uncertainty Quantification Methods for Deep Learningcited by 81
  3. 2023PaSCo: Urban 3D Panoptic Scene Completion with Uncertainty Awarenesscited by 42
  4. 2023Can Unstructured Pruning Reduce the Depth in Deep Neural Networks?cited by 20
  5. 2024LoRA-Ensemble: Efficient Uncertainty Modelling for Self-Attention Networkscited by 20
  6. 2023Window-Based Early-Exit Cascades for Uncertainty Estimation: When Deep Ensembles are More Efficient than Single Modelscited by 16
  7. 2023Make Me a BNN: A Simple Strategy for Estimating Bayesian Uncertainty from Pre-trained Modelscited by 15
  8. 2023A Symmetry-Aware Exploration of Bayesian Neural Network Posteriorscited by 12
  9. 2024Towards Understanding Why Label Smoothing Degrades Selective Classification and How to Fix Itcited by 11
  10. 2024Calib3D: Calibrating Model Preferences for Reliable 3D Scene Understandingcited by 11
  11. 2024Towards Understanding and Quantifying Uncertainty for Text-to-Image Generationcited by 11
  12. 2026Learning Credal Ensembles via Distributionally Robust Optimizationcited by 7
37 more
  1. 2023Are Ensembles Getting Better all the Time?cited by 6
  2. 2024Regularized Multi-Decoder Ensemble for an Error-Aware Scene Representation Networkcited by 6
  3. 2025OCCUQ: Exploring Efficient Uncertainty Quantification for 3D Occupancy Predictioncited by 6
  4. 2026Uncertainty-Aware Reward Modeling for Stable RLHFcited by 6
  5. 2023The Robust Semantic Segmentation UNCV2023 Challenge Resultscited by 5
  6. 2023Tailoring Mixup to Data for Calibrationcited by 5
  7. 2024Uncertainty Estimation for 3D Object Detection via Evidential Learningcited by 5
  8. 2025Ensembling Pruned Attention Heads For Uncertainty-Aware Efficient Transformerscited by 5
  9. 2024Hierarchical Light Transformer Ensembles for Multimodal Trajectory Forecastingcited by 4
  10. 2024Information-driven Affordance Discovery for Efficient Robotic Manipulationcited by 4
  11. 2025Torch-Uncertainty: A Deep Learning Framework for Uncertainty Quantificationcited by 4
  12. 2026Is BatchEnsemble a Single Model? On Calibration and Diversity of Efficient Ensemblescited by 4
  13. 2025Audio-Driven Talking Face Video Generation with Joint Uncertainty Learningcited by 3
  14. 2025Improving Semantic Uncertainty Quantification in LVLMs with Semantic Gaussian Processescited by 3
  15. 2023Learning to Generate Training Datasets for Robust Semantic Segmentationcited by 2
  16. 2025How does Bayesian Sampling help Membership Inference Attacks?cited by 2
  17. 2026A Closer Look at Cross-Domain Few-Shot Object Detection: Fine-Tuning Matters and Parallel Decoder Helpscited by 2
  18. 2024Network Fission Ensembles for Low-Cost Self-Ensemblescited by 1
  19. 2024Low-Cost Self-Ensembles Based on Multi-Branch Transformation and Grouped Convolutioncited by 1
  20. 2025Last Layer Hamiltonian Monte Carlocited by 1
  21. 2025Knowledge Distillation of Uncertainty using Deep Latent Factor Modelcited by 1
  22. 2026Set-based v.s. Distribution-based Representations of Epistemic Uncertainty: A Comparative Studycited by 1
  23. 2026Efficient Credal Prediction through Decalibrationcited by 1
  24. 2026SciDocBench: A Workflow-Centered Benchmark and Data Pipeline for Scientific Document Understandingcited by 1
  25. 2024Awareness of uncertainty in classification using a multivariate model and multi-viewscited by 0
  26. 2025Evidential Uncertainty Probes for Graph Neural Networkscited by 0
  27. 2025Latent Uncertainty Representations for Video-based Driver Action and Intention Recognitioncited by 0
  28. 2025How (Mis)calibrated is your Federated CLIP and what to do about it?cited by 0
  29. 2026On the Equivalence of Random Network Distillation, Deep Ensembles, and Bayesian Inferencecited by 0
  30. 2026UA-Net: Uncertainty-Aware Network for TRISO Image Semantic Segmentationcited by 0
  31. 2026TabSurv: Adapting Modern Tabular Neural Networks to Survival Analysiscited by 0
  32. 2026Multi-Narrow Transformation as a Single-Model Ensemble: Boundary Conditions, Mechanisms, and Failure Modescited by 0
  33. 2026U-SEG: Uncertainty in SEGmentation -- A systematic multi-variable explorationcited by 0
  34. 2026Leveraging Visual Signals for Robust Token-Level Uncertainty in Vision-Language Generationcited by 0
  35. 2026InFlux++: Real and Synthetic Data for Estimating Dynamic Camera Intrinsicscited by 0
  36. 2026Controllable Diversity in Normalization-Based Implicit Ensembles via Softmax-Temperature Modulationcited by 0
  37. 2026Breaking Diversity Collapse in Spiking Pseudo-Ensembles for Efficient OOD Detection in Remote Sensingcited by 0