# Estimating Model Uncertainty of Neural Networks in Sparse Information Form

arXiv: https://arxiv.org/abs/2006.11631 · HTML: https://mlsanity.com/paper/2006.11631/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Jun 2020.

## Built on by (most cited first, 30 of 31)

- [A Survey of Uncertainty in Deep Neural Networks](https://mlsanity.com/paper/2107.03342/) (2021)
- [Laplace Redux -- Effortless Bayesian Deep Learning](https://mlsanity.com/paper/2106.14806/) (2021)
- [LUQ: Long-text Uncertainty Quantification for LLMs](https://mlsanity.com/paper/2403.20279/) (2024)
- [On the Practicality of Deterministic Epistemic Uncertainty](https://mlsanity.com/paper/2107.00649/) (2021)
- [NoiseGrad: Enhancing Explanations by Introducing Stochasticity to Model Weights](https://mlsanity.com/paper/2106.10185/) (2021)
- [A Comprehensive Survey on Evidential Deep Learning and Its Applications](https://mlsanity.com/paper/2409.04720/) (2024)
- [A Survey on Epistemic (Model) Uncertainty in Supervised Learning: Recent Advances and Applications](https://mlsanity.com/paper/2111.01968/) (2021)
- [FedLPA: One-shot Federated Learning with Layer-Wise Posterior Aggregation](https://mlsanity.com/paper/2310.00339/) (2023)
- [Explaining Bayesian Neural Networks](https://mlsanity.com/paper/2108.10346/) (2021)
- [Evaluating Uncertainty-based Failure Detection for Closed-Loop LLM Planners](https://mlsanity.com/paper/2406.00430/) (2024)
- [Trust Your Robots! Predictive Uncertainty Estimation of Neural Networks with Sparse Gaussian Processes](https://mlsanity.com/paper/2109.09690/) (2021)
- [Beyond Unimodal: Generalising Neural Processes for Multimodal Uncertainty Estimation](https://mlsanity.com/paper/2304.01518/) (2023)
- [Uncertainty Estimation for Multi-view Data: The Power of Seeing the Whole Picture](https://mlsanity.com/paper/2210.02676/) (2022)
- [Epistemic Uncertainty Quantification For Pre-trained Neural Network](https://mlsanity.com/paper/2404.10124/) (2024)
- [Bayesian Active Learning for Sim-to-Real Robotic Perception](https://mlsanity.com/paper/2109.11547/) (2021)
- [Uncertainty Quantification for Molecular Property Predictions with Graph Neural Architecture Search](https://mlsanity.com/paper/2307.10438/) (2023)
- [Topology-Matching Normalizing Flows for Out-of-Distribution Detection in Robot Learning](https://mlsanity.com/paper/2311.06481/) (2023)
- [Fast Fishing: Approximating BAIT for Efficient and Scalable Deep Active Image Classification](https://mlsanity.com/paper/2404.08981/) (2024)
- [Learning Expressive Priors for Generalization and Uncertainty Estimation in Neural Networks](https://mlsanity.com/paper/2307.07753/) (2023)
- [Martingale Posterior Neural Networks for Fast Sequential Decision Making](https://mlsanity.com/paper/2506.11898/) (2025)
- [TEDL: A Two-stage Evidential Deep Learning Method for Classification Uncertainty Quantification](https://mlsanity.com/paper/2209.05522/) (2022)
- [Density-based Feasibility Learning with Normalizing Flows for Introspective Robotic Assembly](https://mlsanity.com/paper/2307.01317/) (2023)
- [Low Rank Based Subspace Inference for the Laplace Approximation of Bayesian Neural Networks](https://mlsanity.com/paper/2502.02345/) (2025)
- [Density Regression and Uncertainty Quantification with Bayesian Deep Noise Neural Networks](https://mlsanity.com/paper/2206.05643/) (2022)
- [Virtual Reality via Object Pose Estimation and Active Learning: Realizing Telepresence Robots with Aerial Manipulation Capabilities](https://mlsanity.com/paper/2210.09678/) (2022)
- [SPIRIT: Perceptive Shared Autonomy for Robust Robotic Manipulation under Deep Learning Uncertainty](https://mlsanity.com/paper/2603.05111/) (2026)
- [Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations](https://mlsanity.com/paper/2307.06424/) (2023)
- [Addressing the Inconsistency in Bayesian Deep Learning via Generalized Laplace Approximation](https://mlsanity.com/paper/2405.13535/) (2024)
- [Stein Variational Newton Neural Network Ensembles](https://mlsanity.com/paper/2411.01887/) (2024)
- [Self-Supervised Laplace Approximation for Bayesian Uncertainty Quantification](https://mlsanity.com/paper/2605.12208/) (2026)

## Abstract (arXiv)

We present a sparse representation of model uncertainty for Deep Neural Networks (DNNs) where the parameter posterior is approximated with an inverse formulation of the Multivariate Normal Distribution (MND), also known as the information form. The key insight of our work is that the information matrix, i.e. the inverse of the covariance matrix tends to be sparse in its spectrum. Therefore, dimensionality reduction techniques such as low rank approximations (LRA) can be effectively exploited. To achieve this, we develop a novel sparsification algorithm and derive a cost-effective analytical sampler. As a result, we show that the information form can be scalably applied to represent model uncertainty in DNNs. Our exhaustive theoretical analysis and empirical evaluations on various benchmarks show the competitiveness of our approach over the current methods.