# TIC-TAC: A Framework for Improved Covariance Estimation in Deep Heteroscedastic Regression

arXiv: https://arxiv.org/abs/2310.18953 · HTML: https://mlsanity.com/paper/2310.18953/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 45 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Builds on (most cited first, 27 of 27)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Simple and Scalable Predictive Uncertainty Estimation using Deep Ensembles](https://mlsanity.com/paper/1612.01474/) (2016)
- [What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?](https://mlsanity.com/paper/1703.04977/) (2017)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Deep Bayesian Active Learning with Image Data](https://mlsanity.com/paper/1703.02910/) (2017)
- [ViTPose: Simple Vision Transformer Baselines for Human Pose Estimation](https://mlsanity.com/paper/2204.12484/) (2022)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [PifPaf: Composite Fields for Human Pose Estimation](https://mlsanity.com/paper/1903.06593/) (2019)
- [On the Pitfalls of Heteroscedastic Uncertainty Estimation with Probabilistic Neural Networks](https://mlsanity.com/paper/2203.09168/) (2022)
- [Reliable training and estimation of variance networks](https://mlsanity.com/paper/1906.03260/) (2019)
- [Bayesian Model Selection, the Marginal Likelihood, and Generalization](https://mlsanity.com/paper/2202.11678/) (2022)
- [OpenPifPaf: Composite Fields for Semantic Keypoint Detection and Spatio-Temporal Association](https://mlsanity.com/paper/2103.02440/) (2021)
- [MonoLoco: Monocular 3D Pedestrian Localization and Uncertainty Estimation](https://mlsanity.com/paper/1906.06059/) (2019)
- [Faithful Heteroscedastic Regression with Neural Networks](https://mlsanity.com/paper/2212.09184/) (2022)
- [Structured Uncertainty Prediction Networks](https://mlsanity.com/paper/1802.07079/) (2018)
- [Multivariate Uncertainty in Deep Learning](https://mlsanity.com/paper/1910.14215/) (2019)
- [Few-shot Keypoint Detection with Uncertainty Learning for Unseen Species](https://mlsanity.com/paper/2112.06183/) (2021)
- [Localization with Sampling-Argmax](https://mlsanity.com/paper/2110.08825/) (2021)
- [VL4Pose: Active Learning Through Out-Of-Distribution Detection For Pose Estimation](https://mlsanity.com/paper/2210.06028/) (2022)
- [Learning Structured Gaussians to Approximate Deep Ensembles](https://mlsanity.com/paper/2203.15485/) (2022)
- [Bayesian Uncertainty and Expected Gradient Length -- Regression: Two Sides Of The Same Coin?](https://mlsanity.com/paper/2104.09493/) (2021)
- [Understanding Pose and Appearance Disentanglement in 3D Human Pose Estimation](https://mlsanity.com/paper/2309.11667/) (2023)

## Built on by (most cited first, 4 of 4)

- [MotionMap: Representing Multimodality in Human Pose Forecasting](https://mlsanity.com/paper/2412.18883/) (2024)
- [Towards Self-Supervised Covariance Estimation in Deep Heteroscedastic Regression](https://mlsanity.com/paper/2502.10587/) (2025)
- [Cooperative Variance Estimation and Bayesian Neural Networks for Disentangling Aleatoric and Epistemic Uncertainties](https://mlsanity.com/paper/2505.02743/) (2025)
- [On the Effect of Regularization on Nonparametric Mean-Variance Regression](https://mlsanity.com/paper/2511.22004/) (2025)

## Abstract (arXiv)

Deep heteroscedastic regression involves jointly optimizing the mean and covariance of the predicted distribution using the negative log-likelihood. However, recent works show that this may result in sub-optimal convergence due to the challenges associated with covariance estimation. While the literature addresses this by proposing alternate formulations to mitigate the impact of the predicted covariance, we focus on improving the predicted covariance itself. We study two questions: (1) Does the predicted covariance truly capture the randomness of the predicted mean? (2) In the absence of supervision, how can we quantify the accuracy of covariance estimation? We address (1) with a Taylor Induced Covariance (TIC), which captures the randomness of the predicted mean by incorporating its gradient and curvature through the second order Taylor polynomial. Furthermore, we tackle (2) by introducing a Task Agnostic Correlations (TAC) metric, which combines the notion of correlations and absolute error to evaluate the covariance. We evaluate TIC-TAC across multiple experiments spanning synthetic and real-world datasets. Our results show that not only does TIC accurately learn the covariance, it additionally facilitates an improved convergence of the negative log-likelihood. Our code is available at https://github.com/vita-epfl/TIC-TAC