Unsupervised Joint Image Transfer and Uncertainty Quantification Using Patch Invariant Networks
Not on the map. Why?
Abstract (arXiv)
Unsupervised image transfer enables intra- and inter-modality image translation in applications where a large amount of paired training data is not abundant. To ensure a structure-preserving mapping from the input to the target domain, existing methods for unpaired image transfer are commonly based on cycle-consistency, causing additional computational resources and instability due to the learning of an inverse mapping. This paper presents a novel method for uni-directional domain mapping that does not rely on any paired training data. A proper transfer is achieved by using a GAN architecture and a novel generator loss based on patch invariance. To be more specific, the generator outputs are evaluated and compared at different scales, also leading to an increased focus on high-frequency details as well as an implicit data augmentation. This novel patch loss also offers the possibility to accurately predict aleatoric uncertainty by modeling an input-dependent scale map for the patch residuals. The proposed method is comprehensively evaluated on three well-established medical databases. As compared to four state-of-the-art methods, we observe significantly higher accuracy on these datasets, indicating great potential of the proposed method for unpaired image transfer with uncertainty taken into account. Implementation of the proposed framework is released here: urlhttps://github.com/anger-man/unsupervised-image-transfer-and-uq.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 9 Jul 2022.
Builds on
27 of 38 references.
References · 27
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022Generative Adversarial Networkscited by 5,603
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2017What Uncertainties Do We Need in Bayesian Deep Learning for Computer Vision?cited by 2,335
- 2017Improved Training of Wasserstein GANscited by 2,223
- 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
- 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
15 more
- 2016Least Squares Generative Adversarial Networkscited by 1,000
- 2020A Review of Uncertainty Quantification in Deep Learning: Techniques, Applications and Challengescited by 813
- 2020Contrastive Learning for Unpaired Image-to-Image Translationcited by 637
- 2018fastMRI: An Open Dataset and Benchmarks for Accelerated MRIcited by 464
- 2018MedGAN: Medical Image Translation using GANscited by 152
- 2018Geometry-Consistent Generative Adversarial Networks for One-Sided Unsupervised Domain Mappingcited by 113
- 2017One-Sided Unsupervised Domain Mappingcited by 91
- 2017Deep learning for undersampled MRI reconstructioncited by 91
- 2017Deep MR to CT Synthesis using Unpaired Datacited by 86
- 2018Cross-modality image synthesis from unpaired data using CycleGAN: Effects of gradient consistency loss and training data sizecited by 42
- 2017Bayesian GANcited by 31
- 2018Unpaired Brain MR-to-CT Synthesis using a Structure-Constrained CycleGANcited by 31
- 2021Robustness via Uncertainty-aware Cycle Consistencycited by 16
- 2021Uncertainty-aware GAN with Adaptive Loss for Robust MRI Image Enhancementcited by 15
- 2022Unsupervised Single-shot Depth Estimation using Perceptual Reconstructioncited by 2
Built on by
Citing papers per year: 2023: 2.
2 papers.