# Distribution Matching Losses Can Hallucinate Features in Medical Image Translation

arXiv: https://arxiv.org/abs/1805.08841 · HTML: https://mlsanity.com/paper/1805.08841/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2018.

## Built on by (most cited first, 30 of 99)

- [Deblurring via Stochastic Refinement](https://mlsanity.com/paper/2112.02475/) (2021)
- [Inversion by Direct Iteration: An Alternative to Denoising Diffusion for Image Restoration](https://mlsanity.com/paper/2303.11435/) (2023)
- [Structure-Preserving Multi-Domain Stain Color Augmentation using Style-Transfer with Disentangled Representations](https://mlsanity.com/paper/2107.12357/) (2021)
- [Hallucination Index: An Image Quality Metric for Generative Reconstruction Models](https://mlsanity.com/paper/2407.12780/) (2024)
- [Segmentation-Renormalized Deep Feature Modulation for Unpaired Image Harmonization](https://mlsanity.com/paper/2102.06315/) (2021)
- [A Survey on Training Challenges in Generative Adversarial Networks for Biomedical Image Analysis](https://mlsanity.com/paper/2201.07646/) (2022)
- [BayesCap: Bayesian Identity Cap for Calibrated Uncertainty in Frozen Neural Networks](https://mlsanity.com/paper/2207.06873/) (2022)
- [A Structural Causal Model for MR Images of Multiple Sclerosis](https://mlsanity.com/paper/2103.03158/) (2021)
- [COSMOS: Cross-Modality Unsupervised Domain Adaptation for 3D Medical Image Segmentation based on Target-aware Domain Translation and Iterative Self-Training](https://mlsanity.com/paper/2203.16557/) (2022)
- [Similarity and Quality Metrics for MR Image-To-Image Translation](https://mlsanity.com/paper/2405.08431/) (2024)
- [Low-field magnetic resonance image enhancement via stochastic image quality transfer](https://mlsanity.com/paper/2304.13385/) (2023)
- [Known Operator Learning and Hybrid Machine Learning in Medical Imaging -- A Review of the Past, the Present, and the Future](https://mlsanity.com/paper/2108.04543/) (2021)
- [StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image Translation](https://mlsanity.com/paper/2403.20142/) (2024)
- [Mutually improved endoscopic image synthesis and landmark detection in unpaired image-to-image translation](https://mlsanity.com/paper/2107.06941/) (2021)
- [Data synthesis and adversarial networks: A review and meta-analysis in cancer imaging](https://mlsanity.com/paper/2107.09543/) (2021)
- [Denoising diffusion-based MRI to CT image translation enables automated spinal segmentation](https://mlsanity.com/paper/2308.09345/) (2023)
- [Assessing the ability of generative adversarial networks to learn canonical medical image statistics](https://mlsanity.com/paper/2204.12007/) (2022)
- [High-resolution synthesis of high-density breast mammograms: Application to improved fairness in deep learning based mass detection](https://mlsanity.com/paper/2209.09809/) (2022)
- [StainDiffuser: MultiTask Dual Diffusion Model for Virtual Staining](https://mlsanity.com/paper/2403.11340/) (2024)
- [MR-Contrast-Aware Image-to-Image Translations with Generative Adversarial Networks](https://mlsanity.com/paper/2104.01449/) (2021)
- [METGAN: Generative Tumour Inpainting and Modality Synthesis in Light Sheet Microscopy](https://mlsanity.com/paper/2104.10993/) (2021)
- [Appearance Based Deep Domain Adaptation for the Classification of Aerial Images](https://mlsanity.com/paper/2108.07779/) (2021)
- [Digital staining in optical microscopy using deep learning -- a review](https://mlsanity.com/paper/2303.08140/) (2023)
- [Multitask 3D CBCT-to-CT Translation and Organs-at-Risk Segmentation Using Physics-Based Data Augmentation](https://mlsanity.com/paper/2103.05690/) (2021)
- [High Fidelity Deep Learning-based MRI Reconstruction with Instance-wise Discriminative Feature Matching Loss](https://mlsanity.com/paper/2108.12460/) (2021)
- [Bridging the gap between paired and unpaired medical image translation](https://mlsanity.com/paper/2110.08407/) (2021)
- [Class-Guided Image-to-Image Diffusion: Cell Painting from Brightfield Images with Class Labels](https://mlsanity.com/paper/2303.08863/) (2023)
- [Cross-modal tumor segmentation using generative blending augmentation and self training](https://mlsanity.com/paper/2304.01705/) (2023)
- [Simulating Dynamic Tumor Contrast Enhancement in Breast MRI using Conditional Generative Adversarial Networks](https://mlsanity.com/paper/2409.18872/) (2024)
- [Data Models for Dataset Drift Controls in Machine Learning With Optical Images](https://mlsanity.com/paper/2211.02578/) (2022)

## Abstract (arXiv)

This paper discusses how distribution matching losses, such as those used in CycleGAN, when used to synthesize medical images can lead to mis-diagnosis of medical conditions. It seems appealing to use these new image synthesis methods for translating images from a source to a target domain because they can produce high quality images and some even do not require paired data. However, the basis of how these image translation models work is through matching the translation output to the distribution of the target domain. This can cause an issue when the data provided in the target domain has an over or under representation of some classes (e.g. healthy or sick). When the output of an algorithm is a transformed image there are uncertainties whether all known and unknown class labels have been preserved or changed. Therefore, we recommend that these translated images should not be used for direct interpretation (e.g. by doctors) because they may lead to misdiagnosis of patients based on hallucinated image features by an algorithm that matches a distribution. However there are many recent papers that seem as though this is the goal.