mlsanityData to 7 Oct 2026

Density-Regression: Efficient and Distance-Aware Deep Regressor for Uncertainty Estimation under Distribution Shifts

arXiv 2403.05600 ↗First posted 7 Mar 2024cs.LG stat.MLcited by 7

Not on the map. Why?

Abstract (arXiv)

Morden deep ensembles technique achieves strong uncertainty estimation performance by going through multiple forward passes with different models. This is at the price of a high storage space and a slow speed in the inference (test) time. To address this issue, we propose Density-Regression, a method that leverages the density function in uncertainty estimation and achieves fast inference by a single forward pass. We prove it is distance aware on the feature space, which is a necessary condition for a neural network to produce high-quality uncertainty estimation under distribution shifts. Empirically, we conduct experiments on regression tasks with the cubic toy dataset, benchmark UCI, weather forecast with time series, and depth estimation under real-world shifted applications. We show that Density-Regression has competitive uncertainty estimation performance under distribution shifts with modern deep regressors while using a lower model size and a faster inference speed.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 41 of its 65 references that are in the map. First posted on arXiv 7 Mar 2024.

Builds on

41 of 65 references.

References · 41

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017On Calibration of Modern Neural Networkscited by 4,860
  4. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  5. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2016Density estimation using Real NVPcited by 2,029
  8. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  9. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,274
  10. 2018Deep Reinforcement Learning in a Handful of Trials using Probabilistic Dynamics Modelscited by 748
  11. 2018Evidential Deep Learning to Quantify Classification Uncertaintycited by 728
  12. 2019Conformalized Quantile Regressioncited by 620
29 more
  1. 2017Deep Neural Networks as Gaussian Processescited by 617
  2. 2018GPyTorch: Blackbox Matrix-Matrix Gaussian Process Inference with GPU Accelerationcited by 603
  3. 2015Weight Uncertainty in Neural Networkscited by 554
  4. 2019Conformal Prediction Under Covariate Shiftcited by 511
  5. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
  6. 2018Accurate Uncertainties for Deep Learning Using Calibrated Regressioncited by 417
  7. 2020Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awarenesscited by 389
  8. 2021Revisiting the Calibration of Modern Neural Networkscited by 387
  9. 2019Deep Evidential Regressioncited by 354
  10. 2020Calibrating Deep Neural Networks using Focal Losscited by 353
  11. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  12. 2017Concrete Dropoutcited by 227
  13. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  14. 2020Efficient and Scalable Bayesian Neural Nets with Rank-1 Factorscited by 171
  15. 2020Posterior Network: Uncertainty Estimation without OOD Samples via Density-Based Pseudo-Countscited by 163
  16. 2015A Theoretically Grounded Application of Dropout in Recurrent Neural Networkscited by 161
  17. 2018Flipout: Efficient Pseudo-Independent Weight Perturbations on Mini-Batchescited by 133
  18. 2019Distribution Calibration for Regressioncited by 88
  19. 2021On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertaintycited by 87
  20. 2021Exploiting Domain-Specific Features to Enhance Domain Generalizationcited by 86
  21. 2021Uncertainty Baselines: Benchmarks for Uncertainty & Robustness in Deep Learningcited by 81
  22. 2020Beyond Pinball Loss: Quantile Methods for Calibrated Uncertainty Quantificationcited by 77
  23. 2021Correlated Input-Dependent Label Noise in Large-Scale Image Classificationcited by 37
  24. 2022Nonparametric Uncertainty Quantification for Single Deterministic Neural Networkcited by 36
  25. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  26. 2023A Large-Scale Study of Probabilistic Calibration in Neural Network Regressioncited by 26
  27. 2022JAWS: Auditing Predictive Uncertainty Under Covariate Shiftcited by 14
  28. 2023How Reliable is Your Regression Model's Uncertainty Under Real-World Distribution Shifts?cited by 14
  29. 2023Density-Softmax: Efficient Test-time Model for Uncertainty Estimation and Robustness under Distribution Shiftscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 1; 2026: 3.

7 papers.

  1. 2024Variance-Aware Linear UCB with Deep Representation for Neural Contextual Banditscited by 3
  2. 2024Efficient Nearest Neighbor based Uncertainty Estimation for Natural Language Processing Taskscited by 1
  3. 2024A Decision-driven Methodology for Designing Uncertainty-aware AI Self-Assessmentcited by 1
  4. 2025Q-Learning with Shift-Aware Upper Confidence Bound in Non-Stationary Reinforcement Learningcited by 1
  5. 2026Density-Informed Pseudo-Counts for Calibrated Evidential Deep Learningcited by 0
  6. 2026Do Deep Ensembles Actually Capture Uncertainty in Graph Neural Networks?cited by 0
  7. 2026Efficient and Uncertainty-Aware Diffusion Framework for Offline-to-Online Reinforcement Learningcited by 0