mlsanityData to 14 Oct 2026

Reconstruction of Patient-Specific Confounders in AI-based Radiologic Image Interpretation using Generative Pretraining

arXiv 2309.17123 ↗First posted 29 Sep 2023cs.CV cs.LGcited by 0

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Detecting misleading patterns in automated diagnostic assistance systems, such as those powered by Artificial Intelligence, is critical to ensuring their reliability, particularly in healthcare. Current techniques for evaluating deep learning models cannot visualize confounding factors at a diagnostic level. Here, we propose a self-conditioned diffusion model termed DiffChest and train it on a dataset of 515,704 chest radiographs from 194,956 patients from multiple healthcare centers in the United States and Europe. DiffChest explains classifications on a patient-specific level and visualizes the confounding factors that may mislead the model. We found high inter-reader agreement when evaluating DiffChest's capability to identify treatment-related confounders, with Fleiss' Kappa values of 0.8 or higher across most imaging findings. Confounders were accurately captured with 11.1% to 100% prevalence rates. Furthermore, our pretraining process optimized the model to capture the most relevant information from the input radiographs. DiffChest achieved excellent diagnostic accuracy when diagnosing 11 chest conditions, such as pleural effusion and cardiac insufficiency, and at least sufficient diagnostic accuracy for the remaining conditions. Our findings highlight the potential of pretraining based on diffusion models in medical image classification, specifically in providing insights into confounding factors and model robustness.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 49 of its 75 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

49 of 75 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Survey14 Nov
  2. 2022A Morphology Focused Diffusion Probabilistic Model for Synthesis of Histopathology Images27 Sep
  3. 2021DiffuseMorph: Unsupervised Deformable Image Registration Using Diffusion Model9 Dec
  4. 2021Score-based diffusion models for accelerated MRI8 Oct

Other references · 45

  1. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  9. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  10. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  11. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,988
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
33 more
  1. 2015Learning Deep Features for Discriminative Localizationcited by 2,436
  2. 2017Non-local Neural Networkscited by 2,293
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,937
  4. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  5. 2019CheXpert: A Large Chest Radiograph Dataset with Uncertainty Labels and Expert Comparisoncited by 1,423
  6. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,027
  9. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  10. 2015Deep Learning and the Information Bottleneck Principlecited by 802
  11. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 745
  12. 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 734
  13. 2021DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulationcited by 642
  14. 2021Score-based Generative Modeling in Latent Spacecited by 636
  15. 2021Diffusion Autoencoders: Toward a Meaningful and Decodable Representationcited by 449
  16. 2018Assessing Generative Models via Precision and Recallcited by 406
  17. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 379
  18. 2022Diffusion Models already have a Semantic Latent Spacecited by 328
  19. 2019PadChest: A large chest x-ray image dataset with multi-label annotated reportscited by 309
  20. 2021Robust Compressed Sensing MRI with Deep Generative Priorscited by 286
  21. 2019Large Scale Adversarial Representation Learningcited by 239
  22. 2022Dual Diffusion Implicit Bridges for Image-to-Image Translationcited by 213
  23. 2019Causality matters in medical imagingcited by 193
  24. 2021Causality-inspired Single-source Domain Generalization for Medical Image Segmentationcited by 119
  25. 2019Deep Learning for Deepfakes Creation and Detection: A Surveycited by 102
  26. 2022AUC Maximization in the Era of Big Data and AI: A Surveycited by 100
  27. 2017NegBio: a high-performance tool for negation and uncertainty detection in radiology reportscited by 79
  28. 2019Explanation by Progressive Exaggerationcited by 76
  29. 2022Diffusion Deformable Model for 4D Temporal Medical Image Generationcited by 65
  30. 2022Diffusion Adversarial Representation Learning for Self-supervised Vessel Segmentationcited by 55
  31. 2023High Fidelity Image Counterfactuals with Probabilistic Causal Modelscited by 53
  32. 2022Contextual Attention Network: Transformer Meets U-Netcited by 33
  33. 2020Advancing diagnostic performance and clinical usability of neural networks via adversarial training and dual batch normalizationcited by 12

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Improving Performance, Robustness, and Fairness of Radiographic AI Models with Finely-Controllable Synthetic Data
  2. 2024Exploring Foundation Models for Synthetic Medical Imaging: A Study on Chest X-Rays and Fine-Tuning Techniques
  3. 2024DAug: Diffusion-based Channel Augmentation for Radiology Image Retrieval and Classification
  4. 2026The Learnability Gap in Medical Latent Diffusion
  5. 2025A Generative Foundation Model for Chest Radiography
  6. 2025Language-Guided Trajectory Traversal in Disentangled Stable Diffusion Latent Space for Factorized Medical Image Generation
  7. 2022Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains
  8. 2026CONFLUX: A Latent Diffusion Model for 3D Chest-CT Synthesis with RL Post-Training