# Cyclic Generative Adversarial Networks With Congruent Image-Report Generation For Explainable Medical Image Analysis

arXiv: https://arxiv.org/abs/2211.08424 · HTML: https://mlsanity.com/paper/2211.08424/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 34 of its 137 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 30 of 34)

- [CheXNet: Radiologist-Level Pneumonia Detection on Chest X-Rays with Deep Learning](https://mlsanity.com/paper/1711.05225/) (2017)
- [Efficient Active Learning for Image Classification and Segmentation using a Sample Selection and Conditional Generative Adversarial Network](https://mlsanity.com/paper/1806.05473/) (2018)
- [Chest X-rays Classification: A Multi-Label and Fine-Grained Problem](https://mlsanity.com/paper/1807.07247/) (2018)
- [Relational Subsets Knowledge Distillation for Long-tailed Retinal Diseases Recognition](https://mlsanity.com/paper/2104.11057/) (2021)
- [Structure Preserving Stain Normalization of Histopathology Images Using Self-Supervised Semantic Guidance](https://mlsanity.com/paper/2008.02101/) (2020)
- [Interpretability-Driven Sample Selection Using Self Supervised Learning For Disease Classification And Segmentation](https://mlsanity.com/paper/2104.06087/) (2021)
- [Joint Segmentation and Uncertainty Visualization of Retinal Layers in Optical Coherence Tomography Images using Bayesian Deep Learning](https://mlsanity.com/paper/1809.04282/) (2018)
- [Pathological Retinal Region Segmentation From OCT Images Using Geometric Relation Based Augmentation](https://mlsanity.com/paper/2003.14119/) (2020)
- [Improving Medical Image Classification with Label Noise Using Dual-uncertainty Estimation](https://mlsanity.com/paper/2103.00528/) (2021)
- [Deep multiscale convolutional feature learning for weakly supervised localization of chest pathologies in X-ray images](https://mlsanity.com/paper/1808.08280/) (2018)
- [Night Time Haze and Glow Removal using Deep Dilated Convolutional Network](https://mlsanity.com/paper/1902.00855/) (2019)
- [Progressive Generative Adversarial Networks for Medical Image Super resolution](https://mlsanity.com/paper/1902.02144/) (2019)
- [Adversarial Pulmonary Pathology Translation for Pairwise Chest X-ray Data Augmentation](https://mlsanity.com/paper/1910.04961/) (2019)
- [Registration of Histopathogy Images Using Structural Information From Fine Grained Feature Maps](https://mlsanity.com/paper/2007.02078/) (2020)
- [Informative sample generation using class aware generative adversarial networks for classification of chest Xrays](https://mlsanity.com/paper/1904.10781/) (2019)
- [Probabilistic Integration of Object Level Annotations in Chest X-ray Classification](https://mlsanity.com/paper/2210.06980/) (2022)
- [Retinal Vasculature Segmentation Using Local Saliency Maps and Generative Adversarial Networks For Image Super Resolution](https://mlsanity.com/paper/1710.04783/) (2017)
- [GAN Based Medical Image Registration](https://mlsanity.com/paper/1805.02369/) (2018)
- [Anomaly Detection on X-Rays Using Self-Supervised Aggregation Learning](https://mlsanity.com/paper/2010.09856/) (2020)
- [Consensus Based Medical Image Segmentation Using Semi-Supervised Learning And Graph Cuts](https://mlsanity.com/paper/1612.02166/) (2016)
- [Training Data Independent Image Registration With GANs Using Transfer Learning And Segmentation Information](https://mlsanity.com/paper/1903.10139/) (2019)
- [AMD Severity Prediction And Explainability Using Image Registration And Deep Embedded Clustering](https://mlsanity.com/paper/1907.03075/) (2019)
- [sZoom: A Framework for Automatic Zoom into High Resolution Surveillance Videos](https://mlsanity.com/paper/1909.10164/) (2019)
- [CT Image Synthesis Using Weakly Supervised Segmentation and Geometric Inter-Label Relations For COVID Image Analysis](https://mlsanity.com/paper/2106.10230/) (2021)
- [Generative Adversarial Networks And Domain Adaptation For Training Data Independent Image Registration](https://mlsanity.com/paper/1910.08593/) (2019)
- [Multi-scale Deep Learning Architecture for Nucleus Detection in Renal Cell Carcinoma Microscopy Image](https://mlsanity.com/paper/2104.13557/) (2021)
- [Gated Fusion Network for SAO Filter and Inter Frame Prediction in Versatile Video Coding](https://mlsanity.com/paper/2105.12229/) (2021)
- [Anomaly Detection in Retinal Images using Multi-Scale Deep Feature Sparse Coding](https://mlsanity.com/paper/2201.11506/) (2022)
- [Learning of Inter-Label Geometric Relationships Using Self-Supervised Learning: Application To Gleason Grade Segmentation](https://mlsanity.com/paper/2110.00404/) (2021)
- [Outlier-based Autism Detection using Longitudinal Structural MRI](https://mlsanity.com/paper/2202.09988/) (2022)

## Abstract (arXiv)

We present a novel framework for explainable labeling and interpretation of medical images. Medical images require specialized professionals for interpretation, and are explained (typically) via elaborate textual reports. Different from prior methods that focus on medical report generation from images or vice-versa, we novelly generate congruent image--report pairs employing a cyclic-Generative Adversarial Network (cycleGAN); thereby, the generated report will adequately explain a medical image, while a report-generated image that effectively characterizes the text visually should (sufficiently) resemble the original. The aim of the work is to generate trustworthy and faithful explanations for the outputs of a model diagnosing chest x-ray images by pointing a human user to similar cases in support of a diagnostic decision. Apart from enabling transparent medical image labeling and interpretation, we achieve report and image-based labeling comparable to prior methods, including state-of-the-art performance in some cases as evidenced by experiments on the Indiana Chest X-ray dataset