# VALD-MD: Visual Attribution via Latent Diffusion for Medical Diagnostics

arXiv: https://arxiv.org/abs/2401.01414 · HTML: https://mlsanity.com/paper/2401.01414/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 29 of its 53 references that are in the map.
- First posted on arXiv 2 Jan 2024.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 29 of 29)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [BioBERT: a pre-trained biomedical language representation model for biomedical text mining](https://mlsanity.com/paper/1901.08746/) (2019)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparison](https://mlsanity.com/paper/1901.07031/) (2019)
- [Towards A Rigorous Science of Interpretable Machine Learning](https://mlsanity.com/paper/1702.08608/) (2017)
- [Brain Imaging Generation with Latent Diffusion Models](https://mlsanity.com/paper/2209.07162/) (2022)
- [Diffusion Models for Medical Anomaly Detection](https://mlsanity.com/paper/2203.04306/) (2022)
- [Can AI help in screening Viral and COVID-19 pneumonia?](https://mlsanity.com/paper/2003.13145/) (2020)
- [Unsupervised Medical Image Translation with Adversarial Diffusion Models](https://mlsanity.com/paper/2207.08208/) (2022)
- [Editing Implicit Assumptions in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.08084/) (2023)
- [RoentGen: Vision-Language Foundation Model for Chest X-ray Generation](https://mlsanity.com/paper/2211.12737/) (2022)
- [Diffusion Causal Models for Counterfactual Estimation](https://mlsanity.com/paper/2202.10166/) (2022)
- [Does CLIP Benefit Visual Question Answering in the Medical Domain as Much as it Does in the General Domain?](https://mlsanity.com/paper/2112.13906/) (2021)
- [MDNet: A Semantically and Visually Interpretable Medical Image Diagnosis Network](https://mlsanity.com/paper/1707.02485/) (2017)
- [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022)
- [Visual Feature Attribution using Wasserstein GANs](https://mlsanity.com/paper/1711.08998/) (2017)
- [STEEX: Steering Counterfactual Explanations with Semantics](https://mlsanity.com/paper/2111.09094/) (2021)
- [Deep Multi-instance Networks with Sparse Label Assignment for Whole Mammogram Classification](https://mlsanity.com/paper/1612.05968/) (2016)
- [An Adversarial Learning Approach to Medical Image Synthesis for Lesion Detection](https://mlsanity.com/paper/1810.10850/) (2018)
- [The Swiss Army Knife for Image-to-Image Translation: Multi-Task Diffusion Models](https://mlsanity.com/paper/2204.02641/) (2022)
- [Discriminative Localization in CNNs for Weakly-Supervised Segmentation of Pulmonary Nodules](https://mlsanity.com/paper/1707.01086/) (2017)
- [Robust Tumor Localization with Pyramid Grad-CAM](https://mlsanity.com/paper/1805.11393/) (2018)
- [MED-TEX: Transferring and Explaining Knowledge with Less Data from Pretrained Medical Imaging Models](https://mlsanity.com/paper/2008.02593/) (2020)

## Built on by (most cited first, 2 of 2)

- [Reimagining Anomalies: What If Anomalies Were Normal?](https://mlsanity.com/paper/2402.14469/) (2024)
- [Explainability in Generative Medical Diffusion Models: A Faithfulness-Based Analysis on MRI Synthesis](https://mlsanity.com/paper/2602.09781/) (2026)

## Abstract (arXiv)

Visual attribution in medical imaging seeks to make evident the diagnostically-relevant components of a medical image, in contrast to the more common detection of diseased tissue deployed in standard machine vision pipelines (which are less straightforwardly interpretable/explainable to clinicians). We here present a novel generative visual attribution technique, one that leverages latent diffusion models in combination with domain-specific large language models, in order to generate normal counterparts of abnormal images. The discrepancy between the two hence gives rise to a mapping indicating the diagnostically-relevant image components. To achieve this, we deploy image priors in conjunction with appropriate conditioning mechanisms in order to control the image generative process, including natural language text prompts acquired from medical science and applied radiology. We perform experiments and quantitatively evaluate our results on the COVID-19 Radiography Database containing labelled chest X-rays with differing pathologies via the Frechet Inception Distance (FID), Structural Similarity (SSIM) and Multi Scale Structural Similarity Metric (MS-SSIM) metrics obtained between real and generated images. The resulting system also exhibits a range of latent capabilities including zero-shot localized disease induction, which are evaluated with real examples from the cheXpert dataset.