# ZigZag: Universal Sampling-free Uncertainty Estimation Through Two-Step Inference

arXiv: https://arxiv.org/abs/2211.11435 · HTML: https://mlsanity.com/paper/2211.11435/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 86 references that are in the map.
- First posted on arXiv 21 Nov 2022.

## Builds on (most cited first, 30 of 52)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Evidential Deep Learning to Quantify Classification Uncertainty](https://mlsanity.com/paper/1806.01768/) (2018)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Masked Label Prediction: Unified Message Passing Model for Semi-Supervised Classification](https://mlsanity.com/paper/2009.03509/) (2020)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [Age Progression/Regression by Conditional Adversarial Autoencoder](https://mlsanity.com/paper/1702.08423/) (2017)
- [Lipschitz regularity of deep neural networks: analysis and efficient estimation](https://mlsanity.com/paper/1805.10965/) (2018)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Invertible Residual Networks](https://mlsanity.com/paper/1811.00995/) (2018)
- [Deep Evidential Regression](https://mlsanity.com/paper/1910.02600/) (2019)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)

## Built on by (most cited first, 15 of 15)

- [MirrorCheck: Efficient Adversarial Defense for Vision-Language Models](https://mlsanity.com/paper/2406.09250/) (2024)
- [Beyond Pixels: Semi-Supervised Semantic Segmentation with a Multi-scale Patch-based Multi-Label Classifier](https://mlsanity.com/paper/2407.04036/) (2024)
- [Enabling Uncertainty Estimation in Iterative Neural Networks](https://mlsanity.com/paper/2403.16732/) (2024)
- [Provable Uncertainty Decomposition via Higher-Order Calibration](https://mlsanity.com/paper/2412.18808/) (2024)
- [Assessing Sample Quality via the Latent Space of Generative Models](https://mlsanity.com/paper/2407.15171/) (2024)
- [IT3: Idempotent Test-Time Training](https://mlsanity.com/paper/2410.04201/) (2024)
- [Shadow Removal Refinement via Material-Consistent Shadow Edges](https://mlsanity.com/paper/2409.06848/) (2024)
- [Uncertainty Estimation for 3D Object Detection via Evidential Learning](https://mlsanity.com/paper/2410.23910/) (2024)
- [Weighting Pseudo-Labels via High-Activation Feature Index Similarity and Object Detection for Semi-Supervised Segmentation](https://mlsanity.com/paper/2407.12630/) (2024)
- [High Resolution UDF Meshing via Iterative Networks](https://mlsanity.com/paper/2509.17212/) (2025)
- [Last Layer Hamiltonian Monte Carlo](https://mlsanity.com/paper/2507.08905/) (2025)
- [CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation](https://mlsanity.com/paper/2511.17755/) (2025)
- [Do you understand epistemic uncertainty? Think again! Rigorous frequentist epistemic uncertainty estimation in regression](https://mlsanity.com/paper/2503.13317/) (2025)
- [Latent Uncertainty Representations for Video-based Driver Action and Intention Recognition](https://mlsanity.com/paper/2510.05006/) (2025)
- [GroupEnsemble: Efficient Uncertainty Estimation for DETR-based Object Detection](https://mlsanity.com/paper/2603.01847/) (2026)

## Abstract (arXiv)

Whereas the ability of deep networks to produce useful predictions has been amply demonstrated, estimating the reliability of these predictions remains challenging. Sampling approaches such as MC-Dropout and Deep Ensembles have emerged as the most popular ones for this purpose. Unfortunately, they require many forward passes at inference time, which slows them down. Sampling-free approaches can be faster but suffer from other drawbacks, such as lower reliability of uncertainty estimates, difficulty of use, and limited applicability to different types of tasks and data. In this work, we introduce a sampling-free approach that is generic and easy to deploy, while producing reliable uncertainty estimates on par with state-of-the-art methods at a significantly lower computational cost. It is predicated on training the network to produce the same output with and without additional information about it. At inference time, when no prior information is given, we use the network's own prediction as the additional information. We then take the distance between the predictions with and without prior information as our uncertainty measure. We demonstrate our approach on several classification and regression tasks. We show that it delivers results on par with those of Ensembles but at a much lower computational cost.