# Enabling Uncertainty Estimation in Iterative Neural Networks

arXiv: https://arxiv.org/abs/2403.16732 · HTML: https://mlsanity.com/paper/2403.16732/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 85 references that are in the map.
- First posted on arXiv 25 Mar 2024.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [ShapeNet: An Information-Rich 3D Model Repository](https://mlsanity.com/paper/1512.03012/) (2015)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Fast Graph Representation Learning with PyTorch Geometric](https://mlsanity.com/paper/1903.02428/) (2019)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation](https://mlsanity.com/paper/1511.00561/) (2015)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Predictive Uncertainty Estimation via Prior Networks](https://mlsanity.com/paper/1802.10501/) (2018)
- [Variational Dropout and the Local Reparameterization Trick](https://mlsanity.com/paper/1506.02557/) (2015)
- [Geometric deep learning on graphs and manifolds using mixture model CNNs](https://mlsanity.com/paper/1611.08402/) (2016)
- [Deep Ensembles: A Loss Landscape Perspective](https://mlsanity.com/paper/1912.02757/) (2019)
- [BatchEnsemble: An Alternative Approach to Efficient Ensemble and Lifelong Learning](https://mlsanity.com/paper/2002.06715/) (2020)
- [Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networks](https://mlsanity.com/paper/1502.05336/) (2015)
- [Exploring the Limits of Out-of-Distribution Detection](https://mlsanity.com/paper/2106.03004/) (2021)
- [PAD-Net: Multi-Tasks Guided Prediction-and-Distillation Network for Simultaneous Depth Estimation and Scene Parsing](https://mlsanity.com/paper/1805.04409/) (2018)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Topology-Preserving Deep Image Segmentation](https://mlsanity.com/paper/1906.05404/) (2019)
- [Evaluating Scalable Bayesian Deep Learning Methods for Robust Computer Vision](https://mlsanity.com/paper/1906.01620/) (2019)
- [Human Pose Estimation with Iterative Error Feedback](https://mlsanity.com/paper/1507.06550/) (2015)
- [Generalized Denoising Auto-Encoders as Generative Models](https://mlsanity.com/paper/1305.6663/) (2013)
- [MeshSDF: Differentiable Iso-Surface Extraction](https://mlsanity.com/paper/2006.03997/) (2020)
- [RoadTracer: Automatic Extraction of Road Networks from Aerial Images](https://mlsanity.com/paper/1802.03680/) (2018)
- [Masksembles for Uncertainty Estimation](https://mlsanity.com/paper/2012.08334/) (2020)
- [Bayesian Deep Learning via Subnetwork Inference](https://mlsanity.com/paper/2010.14689/) (2020)

## Built on by (most cited first, 11 of 11)

- [MirrorCheck: Efficient Adversarial Defense for Vision-Language Models](https://mlsanity.com/paper/2406.09250/) (2024)
- [Beyond Pixels: Semi-Supervised Semantic Segmentation with a Multi-scale Patch-based Multi-Label Classifier](https://mlsanity.com/paper/2407.04036/) (2024)
- [Assessing Sample Quality via the Latent Space of Generative Models](https://mlsanity.com/paper/2407.15171/) (2024)
- [IT3: Idempotent Test-Time Training](https://mlsanity.com/paper/2410.04201/) (2024)
- [Shadow Removal Refinement via Material-Consistent Shadow Edges](https://mlsanity.com/paper/2409.06848/) (2024)
- [Uncertainty Estimation for 3D Object Detection via Evidential Learning](https://mlsanity.com/paper/2410.23910/) (2024)
- [Weighting Pseudo-Labels via High-Activation Feature Index Similarity and Object Detection for Semi-Supervised Segmentation](https://mlsanity.com/paper/2407.12630/) (2024)
- [High Resolution UDF Meshing via Iterative Networks](https://mlsanity.com/paper/2509.17212/) (2025)
- [CORA: Consistency-Guided Semi-Supervised Framework for Reasoning Segmentation](https://mlsanity.com/paper/2511.17755/) (2025)
- [C(NN)FD -- Deep Learning Modelling of Multi-Stage Axial Compressors Aerodynamics](https://mlsanity.com/paper/2503.14369/) (2025)
- [Uncertainty Quantification in Federated Granger Causality Learning](https://mlsanity.com/paper/2602.13004/) (2026)

## Abstract (arXiv)

Turning pass-through network architectures into iterative ones, which use their own output as input, is a well-known approach for boosting performance. In this paper, we argue that such architectures offer an additional benefit: The convergence rate of their successive outputs is highly correlated with the accuracy of the value to which they converge. Thus, we can use the convergence rate as a useful proxy for uncertainty. This results in an approach to uncertainty estimation that provides state-of-the-art estimates at a much lower computational cost than techniques like Ensembles, and without requiring any modifications to the original iterative model. We demonstrate its practical value by embedding it in two application domains: road detection in aerial images and the estimation of aerodynamic properties of 2D and 3D shapes.