# LesionDiffusion: Towards Text-controlled General Lesion Synthesis

arXiv: https://arxiv.org/abs/2503.00741 · HTML: https://mlsanity.com/paper/2503.00741/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 2 Mar 2025.

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [The Medical Segmentation Decathlon](https://mlsanity.com/paper/2106.05735/) (2021)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [The KiTS21 Challenge: Automatic segmentation of kidneys, renal tumors, and renal cysts in corticomedullary-phase CT](https://mlsanity.com/paper/2307.01984/) (2023)
- [Towards Generalizable Tumor Synthesis](https://mlsanity.com/paper/2402.19470/) (2024)
- [Label-Free Liver Tumor Segmentation](https://mlsanity.com/paper/2303.14869/) (2023)
- [Label-Free Segmentation of COVID-19 Lesions in Lung CT](https://mlsanity.com/paper/2009.06456/) (2020)
- [CT-Realistic Lung Nodule Simulation from 3D Conditional Generative Adversarial Networks for Robust Lung Segmentation](https://mlsanity.com/paper/1806.04051/) (2018)
- [The state-of-the-art 3D anisotropic intracranial hemorrhage segmentation on non-contrast head CT: The INSTANCE challenge](https://mlsanity.com/paper/2301.03281/) (2023)
- [Unleashing the Potential of Vision-Language Pre-Training for 3D Zero-Shot Lesion Segmentation via Mask-Attribute Alignment](https://mlsanity.com/paper/2410.15744/) (2024)
- [A Synthetic Data-Driven Radiology Foundation Model for Pan-tumor Clinical Diagnosis](https://mlsanity.com/paper/2502.06171/) (2025)
- [CT Synthesis with Conditional Diffusion Models for Abdominal Lymph Node Segmentation](https://mlsanity.com/paper/2403.17770/) (2024)

## Built on by (most cited first, 4 of 4)

- [CLAIM: Clinically-Guided LGE Augmentation for Realistic and Diverse Myocardial Scar Synthesis and Segmentation](https://mlsanity.com/paper/2506.15549/) (2025)
- [Prompt Mechanisms in Medical Imaging: A Comprehensive Survey](https://mlsanity.com/paper/2507.01055/) (2025)
- [Longitudinal Lesion Inpainting in Brain MRI via 3D Region Aware Diffusion](https://mlsanity.com/paper/2603.05693/) (2026)
- [Diffusion Models in Medical Image Inpainting: Challenges, Solution Taxonomy, and Future Directions](https://mlsanity.com/paper/2607.21904/) (2026)

## Abstract (arXiv)

Fully-supervised lesion recognition methods in medical imaging face challenges due to the reliance on large annotated datasets, which are expensive and difficult to collect. To address this, synthetic lesion generation has become a promising approach. However, existing models struggle with scalability, fine-grained control over lesion attributes, and the generation of complex structures. We propose LesionDiffusion, a text-controllable lesion synthesis framework for 3D CT imaging that generates both lesions and corresponding masks. By utilizing a structured lesion report template, our model provides greater control over lesion attributes and supports a wider variety of lesion types. We introduce a dataset of 1,505 annotated CT scans with paired lesion masks and structured reports, covering 14 lesion types across 8 organs. LesionDiffusion consists of two components: a lesion mask synthesis network (LMNet) and a lesion inpainting network (LINet), both guided by lesion attributes and image features. Extensive experiments demonstrate that LesionDiffusion significantly improves segmentation performance, with strong generalization to unseen lesion types and organs, outperforming current state-of-the-art models. Code is available at https://github.com/HengruiTianSJTU/LesionDiffusion.