# Deep Sub-Ensembles for Fast Uncertainty Estimation in Image Classification

arXiv: https://arxiv.org/abs/1910.08168 · HTML: https://mlsanity.com/paper/1910.08168/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Oct 2019.

## Built on by (most cited first, 30 of 36)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [Are Uncertainty Quantification Capabilities of Evidential Deep Learning a Mirage?](https://mlsanity.com/paper/2402.06160/) (2024)
- [Hyperbolic Uncertainty Aware Semantic Segmentation](https://mlsanity.com/paper/2203.08881/) (2022)
- [Gradient-Based Quantification of Epistemic Uncertainty for Deep Object Detectors](https://mlsanity.com/paper/2107.04517/) (2021)
- [Layer Ensembles: A Single-Pass Uncertainty Estimation in Deep Learning for Segmentation](https://mlsanity.com/paper/2203.08878/) (2022)
- [Active Learning of Discrete-Time Dynamics for Uncertainty-Aware Model Predictive Control](https://mlsanity.com/paper/2210.12583/) (2022)
- [Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimation](https://mlsanity.com/paper/2304.01518/) (2023)
- [Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture](https://mlsanity.com/paper/2210.02676/) (2022)
- [Gradient-based Uncertainty Attribution for Explainable Bayesian Deep Learning](https://mlsanity.com/paper/2304.04824/) (2023)
- [SSE: Multimodal Semantic Data Selection and Enrichment for Industrial-scale Data Assimilation](https://mlsanity.com/paper/2409.13860/) (2024)
- [Regularized Multi-Decoder Ensemble for an Error-Aware Scene Representation Network](https://mlsanity.com/paper/2407.19082/) (2024)
- [OCCUQ: Exploring Efficient Uncertainty Quantification for 3D Occupancy Prediction](https://mlsanity.com/paper/2503.10605/) (2025)
- [Compensation Learning in Semantic Segmentation](https://mlsanity.com/paper/2304.13428/) (2023)
- [Building a Winning Team: Selecting Source Model Ensembles using a Submodular Transferability Estimation Approach](https://mlsanity.com/paper/2309.02429/) (2023)
- [TEDL: A Two-stage Evidential Deep Learning Method for Classification Uncertainty Quantification](https://mlsanity.com/paper/2209.05522/) (2022)
- [Layer Ensembles](https://mlsanity.com/paper/2210.04882/) (2022)
- [Uncertainty-Aware AB3DMOT by Variational 3D Object Detection](https://mlsanity.com/paper/2302.05923/) (2023)
- [Efficient Post-Hoc Uncertainty Calibration via Variance-Based Smoothing](https://mlsanity.com/paper/2503.15583/) (2025)
- [Mimir: Hierarchical Goal-Driven Diffusion with Uncertainty Propagation for End-to-End Autonomous Driving](https://mlsanity.com/paper/2512.07130/) (2025)
- [Selective classification using a robust meta-learning approach](https://mlsanity.com/paper/2212.05987/) (2022)
- [Quantifying Uncertainty in the Presence of Distribution Shifts](https://mlsanity.com/paper/2506.18283/) (2025)
- [Intra-Class Probabilistic Embeddings for Uncertainty Estimation in Vision-Language Models](https://mlsanity.com/paper/2511.22019/) (2025)
- [Gated Linear Model induced U-net for surrogate modeling and uncertainty quantification](https://mlsanity.com/paper/2111.05123/) (2021)
- [Transfer learning for ensembles: reducing computation time and keeping the diversity](https://mlsanity.com/paper/2206.13116/) (2022)
- [Secure Deep Learning-based Distributed Intelligence on Pocket-sized Drones](https://mlsanity.com/paper/2307.01559/) (2023)
- [Efficient Credal Prediction through Decalibration](https://mlsanity.com/paper/2603.08495/) (2026)
- [Dropout Prediction Uncertainty Estimation Using Neuron Activation Strength](https://mlsanity.com/paper/2110.06435/) (2021)
- [SoftDropConnect (SDC) -- Effective and Efficient Quantification of the Network Uncertainty in Deep MR Image Analysis](https://mlsanity.com/paper/2201.08418/) (2022)
- [Variational Voxel Pseudo Image Tracking](https://mlsanity.com/paper/2302.05914/) (2023)

## Abstract (arXiv)

Fast estimates of model uncertainty are required for many robust robotics applications. Deep Ensembles provides state of the art uncertainty without requiring Bayesian methods, but still it is computationally expensive. In this paper we propose deep sub-ensembles, an approximation to deep ensembles where the core idea is to ensemble only the layers close to the output, and not the whole model. With ResNet-20 on the CIFAR10 dataset, we obtain 1.5-2.5 speedup over a Deep Ensemble, with a small increase in error and NLL, and similarly up to 5-15 speedup with a VGG-like network on the SVHN dataset. Our results show that this idea enables a trade-off between error and uncertainty quality versus computational performance.