# Multivariate Uncertainty in Deep Learning

arXiv: https://arxiv.org/abs/1910.14215 · HTML: https://mlsanity.com/paper/1910.14215/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2019.

## Built on by (most cited first, 27 of 27)

- [CoordiNet: uncertainty-aware pose regressor for reliable vehicle localization](https://mlsanity.com/paper/2103.10796/) (2021)
- [AutoDEUQ: Automated Deep Ensemble with Uncertainty Quantification](https://mlsanity.com/paper/2110.13511/) (2021)
- [IDOL: Inertial Deep Orientation-Estimation and Localization](https://mlsanity.com/paper/2102.04024/) (2021)
- [MAC-VO: Metrics-aware Covariance for Learning-based Stereo Visual Odometry](https://mlsanity.com/paper/2409.09479/) (2024)
- [Traffic Prediction using Artificial Intelligence: Review of Recent Advances and Emerging Opportunities](https://mlsanity.com/paper/2305.19591/) (2023)
- [CTIN: Robust Contextual Transformer Network for Inertial Navigation](https://mlsanity.com/paper/2112.02143/) (2021)
- [EqNIO: Subequivariant Neural Inertial Odometry](https://mlsanity.com/paper/2408.06321/) (2024)
- [A Survey on Uncertainty Reasoning and Quantification for Decision Making: Belief Theory Meets Deep Learning](https://mlsanity.com/paper/2206.05675/) (2022)
- [AirIMU: Learning Uncertainty Propagation for Inertial Odometry](https://mlsanity.com/paper/2310.04874/) (2023)
- [Learning Visuotactile Estimation and Control for Non-prehensile Manipulation under Occlusions](https://mlsanity.com/paper/2412.13157/) (2024)
- [Swift sky localization of gravitational waves using deep learning seeded importance sampling](https://mlsanity.com/paper/2111.00833/) (2021)
- [TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression](https://mlsanity.com/paper/2310.18953/) (2023)
- [Uncertainty-Aware AB3DMOT by Variational 3D Object Detection](https://mlsanity.com/paper/2302.05923/) (2023)
- [Pedestrian Trajectory Forecasting Using Deep Ensembles Under Sensing Uncertainty](https://mlsanity.com/paper/2305.16620/) (2023)
- [Benchmarking Visual-Inertial Deep Multimodal Fusion for Relative Pose Regression and Odometry-aided Absolute Pose Regression](https://mlsanity.com/paper/2208.00919/) (2022)
- [Informative Data Selection with Uncertainty for Multi-modal Object Detection](https://mlsanity.com/paper/2304.11697/) (2023)
- [Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression](https://mlsanity.com/paper/2502.10587/) (2025)
- [Learning-based methods to model small body gravity fields for proximity operations: Safety and Robustness](https://mlsanity.com/paper/2112.09998/) (2021)
- [Deep Modeling of Non-Gaussian Aleatoric Uncertainty](https://mlsanity.com/paper/2405.20513/) (2024)
- [Uncertainty Propagation for Echocardiography Clinical Metric Estimation via Contour Sampling](https://mlsanity.com/paper/2502.12713/) (2025)
- [Multivariate Uncertainty Quantification with Tomographic Quantile Forests](https://mlsanity.com/paper/2512.16383/) (2025)
- [An End-to-End Learning-Based Multi-Sensor Fusion for Autonomous Vehicle Localization](https://mlsanity.com/paper/2503.05088/) (2025)
- [Aleatoric Uncertainty from AI-based 6D Object Pose Predictors for Object-relative State Estimation](https://mlsanity.com/paper/2509.01583/) (2025)
- [ProTerrain: Probabilistic Physics-Informed Rough Terrain World Modeling](https://mlsanity.com/paper/2510.19364/) (2025)
- [Machine Learning Argument of Latitude Error Model for LEO Satellite Orbit and Covariance Correction](https://mlsanity.com/paper/2602.16764/) (2026)
- [Vision-Based Safe Human-Robot Collaboration with Uncertainty Guarantees](https://mlsanity.com/paper/2604.15221/) (2026)
- [It depends: Incorporating correlations for joint aleatoric and epistemic uncertainties of high-dimensional output spaces](https://mlsanity.com/paper/2608.24518/) (2026)

## Abstract (arXiv)

Deep learning has the potential to dramatically impact navigation and tracking state estimation problems critical to autonomous vehicles and robotics. Measurement uncertainties in state estimation systems based on Kalman and other Bayes filters are typically assumed to be a fixed covariance matrix. This assumption is risky, particularly for "black box" deep learning models, in which uncertainty can vary dramatically and unexpectedly. Accurate quantification of multivariate uncertainty will allow for the full potential of deep learning to be used more safely and reliably in these applications. We show how to model multivariate uncertainty for regression problems with neural networks, incorporating both aleatoric and epistemic sources of heteroscedastic uncertainty. We train a deep uncertainty covariance matrix model in two ways: directly using a multivariate Gaussian density loss function, and indirectly using end-to-end training through a Kalman filter. We experimentally show in a visual tracking problem the large impact that accurate multivariate uncertainty quantification can have on Kalman filter performance for both in-domain and out-of-domain evaluation data. We additionally show in a challenging visual odometry problem how end-to-end filter training can allow uncertainty predictions to compensate for filter weaknesses.