mlsanityData to 7 Oct 2026

MaskMedPaint: Masked Medical Image Inpainting with Diffusion Models for Mitigation of Spurious Correlations

arXiv 2411.10686 ↗First posted 16 Nov 2024cs.CV cs.LGcited by 2

On the map: Diffusion models → Medical imaging and segmentation

Abstract (arXiv)

Spurious features associated with class labels can lead image classifiers to rely on shortcuts that don't generalize well to new domains. This is especially problematic in medical settings, where biased models fail when applied to different hospitals or systems. In such cases, data-driven methods to reduce spurious correlations are preferred, as clinicians can directly validate the modified images. While Denoising Diffusion Probabilistic Models (Diffusion Models) show promise for natural images, they are impractical for medical use due to the difficulty of describing spurious medical features. To address this, we propose Masked Medical Image Inpainting (MaskMedPaint), which uses text-to-image diffusion models to augment training images by inpainting areas outside key classification regions to match the target domain. We demonstrate that MaskMedPaint enhances generalization to target domains across both natural (Waterbirds, iWildCam) and medical (ISIC 2018, Chest X-ray) datasets, given limited unlabeled target images.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. Builds on 30 of its 45 references that are in the map. First posted on arXiv 16 Nov 2024.

Builds on

30 of 45 references.

References · 30

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2016Densely Connected Convolutional Networkscited by 6,691
  4. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  5. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  6. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  7. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  8. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  9. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  10. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  11. 2020WILDS: A Benchmark of in-the-Wild Distribution Shiftscited by 1,366
  12. 2019Distributionally Robust Neural Networks for Group Shifts: On the Importance of Regularization for Worst-Case Generalizationcited by 1,232
18 more
  1. 2020In Search of Lost Domain Generalizationcited by 1,042
  2. 2017ChestX-ray8: Hospital-scale Chest X-ray Database and Benchmarks on Weakly-Supervised Classification and Localization of Common Thorax Diseasescited by 969
  3. 2021Resolution-robust Large Mask Inpainting with Fourier Convolutionscited by 730
  4. 2019Skin Lesion Analysis Toward Melanoma Detection 2018: A Challenge Hosted by the International Skin Imaging Collaboration (ISIC)cited by 657
  5. 2019MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographscited by 590
  6. 2022Paint by Example: Exemplar-based Image Editing with Diffusion Modelscited by 489
  7. 2022Last Layer Re-Training is Sufficient for Robustness to Spurious Correlationscited by 379
  8. 2018Confounding variables can degrade generalization performance of radiological deep learning modelscited by 374
  9. 2019Hidden Stratification Causes Clinically Meaningful Failures in Machine Learning for Medical Imagingcited by 215
  10. 2022Improving Out-of-Distribution Robustness via Selective Augmentationcited by 202
  11. 2017Controllable Invariance through Adversarial Feature Learningcited by 127
  12. 2023Diversify Your Vision Datasets with Automatic Diffusion-Based Augmentationcited by 100
  13. 2021Transparency of Deep Neural Networks for Medical Image Analysis: A Review of Interpretability Methodscited by 83
  14. 2022The ArtBench Dataset: Benchmarking Generative Models with Artworkscited by 64
  15. 2019Can we trust deep learning models diagnosis? The impact of domain shift in chest radiograph classificationcited by 57
  16. 2022UMIX: Improving Importance Weighting for Subpopulation Shift via Uncertainty-Aware Mixupcited by 34
  17. 2022Using Language to Extend to Unseen Domainscited by 34
  18. 2022Spuriosity Rankings: Sorting Data to Measure and Mitigate Biasescited by 13

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025AURAD: Anatomy-Pathology Unified Radiology Synthesis with Progressive Representationscited by 0
  2. 2025HistoLens: An Interactive XAI Toolkit for Verifying and Mitigating Flaws in Vision-Language Models for Histopathologycited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions
  2. 2025Inpainting is All You Need: A Diffusion-based Augmentation Method for Semi-supervised Medical Image Segmentation
  3. 2026Reconstruction-Shift Discrimination via Mask-Guided Latent Diffusion for Medical Anomaly Detection
  4. 2023DiffMIC: Dual-Guidance Diffusion Network for Medical Image Classification
  5. 2024BrushNet: A Plug-and-Play Image Inpainting Model with Decomposed Dual-Branch Diffusion
  6. 2025Conditional Diffusion Models are Medical Image Classifiers that Provide Explainability and Uncertainty for Free
  7. 2022Multitask Brain Tumor Inpainting with Diffusion Models: A Methodological Report
  8. 2026MedDiME: Efficient Latent Diffusion with Adaptive Masking for Medical Counterfactual Generation