# An Interpretable Local Editing Model for Counterfactual Medical Image Generation

arXiv: https://arxiv.org/abs/2603.00423 · HTML: https://mlsanity.com/paper/2603.00423/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 28 Feb 2026.

## Builds on (most cited first, 17 of 17)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Making the Most of Text Semantics to Improve Biomedical Vision--Language Processing](https://mlsanity.com/paper/2204.09817/) (2022)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [TorchXRayVision: A library of chest X-ray datasets and models](https://mlsanity.com/paper/2111.00595/) (2021)
- [Generative models improve fairness of medical classifiers under distribution shifts](https://mlsanity.com/paper/2304.09218/) (2023)
- [Which Explanation Should I Choose? A Function Approximation Perspective to Characterizing Post Hoc Explanations](https://mlsanity.com/paper/2206.01254/) (2022)
- [LLM-CXR: Instruction-Finetuned LLM for CXR Image Understanding and Generation](https://mlsanity.com/paper/2305.11490/) (2023)
- [What is Healthy? Generative Counterfactual Diffusion for Lesion Localization](https://mlsanity.com/paper/2207.12268/) (2022)
- [Watch Your Steps: Local Image and Scene Editing by Text Instructions](https://mlsanity.com/paper/2308.08947/) (2023)
- [Measurable Counterfactual Local Explanations for Any Classifier](https://mlsanity.com/paper/1908.03020/) (2019)
- [Expert Knowledge-Aware Image Difference Graph Representation Learning for Difference-Aware Medical Visual Question Answering](https://mlsanity.com/paper/2307.11986/) (2023)
- [Diffusion Models for Counterfactual Generation and Anomaly Detection in Brain Images](https://mlsanity.com/paper/2308.02062/) (2023)
- [BiomedJourney: Counterfactual Biomedical Image Generation by Instruction-Learning from Multimodal Patient Journeys](https://mlsanity.com/paper/2310.10765/) (2023)
- [RadEdit: stress-testing biomedical vision models via diffusion image editing](https://mlsanity.com/paper/2312.12865/) (2023)
- [DreaMR: Diffusion-driven Counterfactual Explanation for Functional MRI](https://mlsanity.com/paper/2307.09547/) (2023)
- [Mitigating attribute amplification in counterfactual image generation](https://mlsanity.com/paper/2403.09422/) (2024)

## Built on by (most cited first, 1 of 1)

- [Clinically Aware Synthetic Image Generation for Concept Coverage in Chest X-ray Models](https://mlsanity.com/paper/2603.15525/) (2026)

## Abstract (arXiv)

Counterfactual medical image generation have emerged as a critical tool for enhancing AI-driven systems in medical domain by answering "what-if" questions. However, existing approaches face two fundamental limitations: First, they fail to prevent unintended modifications, resulting collateral changes in demographic attributes when only disease features should be affected. Second, they lack interpretability in their editing process, which significantly limits their utility in real-world medical applications. To address these limitations, we present InstructX2X, a novel interpretable local editing model for counterfactual medical image generation featuring Region-Specific Editing. This approach restricts modifications to specific regions, effectively preventing unintended changes while simultaneously providing a Guidance Map that offers inherently interpretable visual explanations of the editing process. Additionally, we introduce MIMIC-EDIT-INSTRUCTION, a dataset for counterfactual medical image generation derived from expert-verified medical VQA pairs. Through extensive experiments, InstructX2X achieve state-of-the-art performance across all major evaluation metrics. Our model successfully generates high-quality counterfactual chest X-ray images along with interpretable explanations.