# LeFusion: Controllable Pathology Synthesis via Lesion-Focused Diffusion Models arXiv: https://arxiv.org/abs/2403.14066 ยท HTML: https://mlsanity.com/paper/2403.14066/ ## Key facts - Cited by 15 papers in the map (data through 14 Oct 2026). - Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models. - Builds on 30 of its 54 references that are in the map. - First posted on arXiv 21 Mar 2024. ## Placed in - Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/) ## Builds on (most cited first, 30 of 30) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022) - [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020) - [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017) - [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021) - [RePaint: Inpainting using Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.09865/) (2022) - [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018) - [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021) - [Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images](https://mlsanity.com/paper/2201.01266/) (2022) - [Deep Long-Tailed Learning: A Survey](https://mlsanity.com/paper/2110.04596/) (2021) - [Rethinking FID: Towards a Better Evaluation Metric for Image Generation](https://mlsanity.com/paper/2401.09603/) (2023) - [SynthSeg: Segmentation of brain MRI scans of any contrast and resolution without retraining](https://mlsanity.com/paper/2107.09559/) (2021) - [Secure and Robust Machine Learning for Healthcare: A Survey](https://mlsanity.com/paper/2001.08103/) (2020) - [Towards Generalizable Tumor Synthesis](https://mlsanity.com/paper/2402.19470/) (2024) - [GenerateCT: Text-Conditional Generation of 3D Chest CT Volumes](https://mlsanity.com/paper/2305.16037/) (2023) - [Label-Free Liver Tumor Segmentation](https://mlsanity.com/paper/2303.14869/) (2023) - [Overcoming Data Scarcity in Biomedical Imaging with a Foundational Multi-Task Model](https://mlsanity.com/paper/2311.09847/) (2023) - [Label-Free Segmentation of COVID-19 Lesions in Lung CT](https://mlsanity.com/paper/2009.06456/) (2020) - [ArSDM: Colonoscopy Images Synthesis with Adaptive Refinement Semantic Diffusion Models](https://mlsanity.com/paper/2309.01111/) (2023) - [Synthesizing Diverse Lung Nodules Wherever Massively: 3D Multi-Conditional GAN-based CT Image Augmentation for Object Detection](https://mlsanity.com/paper/1906.04962/) (2019) - [Deep Learning methods for automatic evaluation of delayed enhancement-MRI. The results of the EMIDEC challenge](https://mlsanity.com/paper/2108.04016/) (2021) - [From Pixel to Cancer: Cellular Automata in Computed Tomography](https://mlsanity.com/paper/2403.06459/) (2024) - [Multitask Brain Tumor Inpainting with Diffusion Models: A Methodological Report](https://mlsanity.com/paper/2210.12113/) (2022) - [Free-form tumor synthesis in computed tomography images via richer generative adversarial network](https://mlsanity.com/paper/2104.09701/) (2021) - [Generative Enhancement for 3D Medical Images](https://mlsanity.com/paper/2403.12852/) (2024) - [Metadata-Conditioned Generative Models to Synthesize Anatomically-Plausible 3D Brain MRIs](https://mlsanity.com/paper/2310.04630/) (2023) - [METGAN: Generative Tumour Inpainting and Modality Synthesis in Light Sheet Microscopy](https://mlsanity.com/paper/2104.10993/) (2021) - [Class-Aware Adversarial Lung Nodule Synthesis in CT Images](https://mlsanity.com/paper/1812.11204/) (2018) ## Built on by (most cited first, 15 of 15) - [CLAIM: Clinically-Guided LGE Augmentation for Realistic and Diverse Myocardial Scar Synthesis and Segmentation](https://mlsanity.com/paper/2506.15549/) (2025) - [GRASP: Guided Residual Adapters with Sample-wise Partitioning](https://mlsanity.com/paper/2512.01675/) (2025) - [Meply: A Large-scale Dataset and Baseline Evaluations for Metastatic Perirectal Lymph Node Detection and Segmentation](https://mlsanity.com/paper/2404.08916/) (2024) - [Exploring the Effectiveness of Deep Features from Domain-Specific Foundation Models in Retinal Image Synthesis](https://mlsanity.com/paper/2506.11753/) (2025) - [USB: Unified Synthetic Brain Framework for Bidirectional Pathology-Healthy Generation and Editing](https://mlsanity.com/paper/2512.00269/) (2025) - [Glance and Focus Reinforcement for Pan-cancer Screening](https://mlsanity.com/paper/2601.19103/) (2026) - [LN-Gen: Rectal Lymph Nodes Generation via Anatomical Features](https://mlsanity.com/paper/2408.14977/) (2024) - [PathoSyn: Imaging-Pathology MRI Synthesis via Disentangled Deviation Diffusion](https://mlsanity.com/paper/2512.23130/) (2025) - [Flow Matching with Optimized Subclass Priors for Medical Image Augmentation](https://mlsanity.com/paper/2605.16469/) (2026) - [Are LLMs Ready to Assist Physicians? PhysAssistBench for Interactive Doctor-Patient-EHR Assistance](https://mlsanity.com/paper/2606.18613/) (2026) - [Local Label-Informed Feature Transfer for Generating Ground-Truth Medical Images: A Comparison of GAN- and Diffusion-Based Approaches](https://mlsanity.com/paper/2607.18882/) (2026) - [Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions](https://mlsanity.com/paper/2607.21904/) (2026) - [VeCAS: Vessel-Focused Contrast-Free Angiogram Synthesis for Vascular Interventions](https://mlsanity.com/paper/2608.22828/) (2026) - [Latent Space Is Not Flat: Rethinking Latent Structure for 3D Medical Image Synthesis](https://mlsanity.com/paper/2609.32794/) (2026) - [PCaPaint: Prostate Cancer Inpainting by Mitigating Shortcut Learning](https://mlsanity.com/paper/2609.37350/) (2026) ## Abstract (arXiv) Patient data from real-world clinical practice often suffers from data scarcity and long-tail imbalances, leading to biased outcomes or algorithmic unfairness. This study addresses these challenges by generating lesion-containing image-segmentation pairs from lesion-free images. Previous efforts in medical imaging synthesis have struggled with separating lesion information from background, resulting in low-quality backgrounds and limited control over the synthetic output. Inspired by diffusion-based image inpainting, we propose LeFusion, a lesion-focused diffusion model. By redesigning the diffusion learning objectives to focus on lesion areas, we simplify the learning process and improve control over the output while preserving high-fidelity backgrounds by integrating forward-diffused background contexts into the reverse diffusion process. Additionally, we tackle two major challenges in lesion texture synthesis: 1) multi-peak and 2) multi-class lesions. We introduce two effective strategies: histogram-based texture control and multi-channel decomposition, enabling the controlled generation of high-quality lesions in difficult scenarios. Furthermore, we incorporate lesion mask diffusion, allowing control over lesion size, location, and boundary, thus increasing lesion diversity. Validated on 3D cardiac lesion MRI and lung nodule CT datasets, LeFusion-generated data significantly improves the performance of state-of-the-art segmentation models, including nnUNet and SwinUNETR. Code and model are available at https://github.com/M3DV/LeFusion.