mlsanityData to 14 Oct 2026

What's Behind the Mask: Estimating Uncertainty in Image-to-Image Problems

arXiv 2211.15211 ↗First posted 28 Nov 2022cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Estimating uncertainty in image-to-image networks is an important task, particularly as such networks are being increasingly deployed in the biological and medical imaging realms. In this paper, we introduce a new approach to this problem based on masking. Given an existing image-to-image network, our approach computes a mask such that the distance between the masked reconstructed image and the masked true image is guaranteed to be less than a specified threshold, with high probability. The mask thus identifies the more certain regions of the reconstructed image. Our approach is agnostic to the underlying image-to-image network, and only requires triples of the input (degraded), reconstructed and true images for training. Furthermore, our method is agnostic to the distance metric used. As a result, one can use Lp-style distances or perceptual distances like LPIPS, which contrasts with interval-based approaches to uncertainty. Our theoretical guarantees derive from a conformal calibration procedure. We evaluate our mask-based approach to uncertainty on image colorization, image completion, and super-resolution tasks, demonstrating high quality performance on each.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 28 of its 37 references that are in the map. First posted on arXiv 28 Nov 2022.

Builds on

28 of 37 references.

References · 28

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  2. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  3. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,926
  4. 2017Deep Bayesian Active Learning with Image Datacited by 819
  5. 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 819
  6. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 792
  7. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  8. 2019Conformalized Quantile Regressioncited by 629
  9. 2014Stochastic Gradient Hamiltonian Monte Carlocited by 343
  10. 2020Hands-on Bayesian Neural Networks -- a Tutorial for Deep Learning Userscited by 338
  11. 2022Conformal Risk Controlcited by 299
  12. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 288
16 more
  1. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 271
  2. 2017Concrete Dropoutcited by 227
  3. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 210
  4. 2017Multiplicative Normalizing Flows for Variational Bayesian Neural Networkscited by 157
  5. 2018High-Quality Prediction Intervals for Deep Learning: A Distribution-Free, Ensembled Approachcited by 117
  6. 2022Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imagingcited by 107
  7. 2019Subspace Inference for Bayesian Deep Learningcited by 95
  8. 2019A comparison of some conformal quantile regression methodscited by 76
  9. 2019Adaptive, Distribution-Free Prediction Intervals for Deep Networkscited by 56
  10. 2020Predictive Inference Is Free with the Jackknife+-after-Bootstrapcited by 55
  11. 2021Quantifying Uncertainty in Deep Spatiotemporal Forecastingcited by 48
  12. 2022Semantic uncertainty intervals for disentangled latent spacescited by 22
  13. 2020Frequentist Uncertainty in Recurrent Neural Networks via Blockwise Influence Functionscited by 18
  14. 2019Variational Inference to Measure Model Uncertainty in Deep Neural Networkscited by 12
  15. 2022Conformal Methods for Quantifying Uncertainty in Spatiotemporal Data: A Surveycited by 11
  16. 2019The MBPEP: a deep ensemble pruning algorithm providing high quality uncertainty predictioncited by 10

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 3.

5 papers.

  1. 2023How to Trust Your Diffusion Model: A Convex Optimization Approach to Conformal Risk Controlcited by 50
  2. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 30
  3. 2024Conformal Prediction: A Data Perspectivecited by 25
  4. 2024Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertaintycited by 20
  5. 2023Aerial Image Object Detection With Vision Transformer Detector (ViTDet)cited by 3