# Medical Image Synthesis via Fine-Grained Image-Text Alignment and Anatomy-Pathology Prompting

arXiv: https://arxiv.org/abs/2403.06835 · HTML: https://mlsanity.com/paper/2403.06835/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 15 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Builds on (most cited first, 4 of 4)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Taming Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2012.09841/) (2020)
- [Adapting Pretrained Vision-Language Foundational Models to Medical Imaging Domains](https://mlsanity.com/paper/2210.04133/) (2022)
- [SkrGAN: Sketching-rendering Unconditional Generative Adversarial Networks for Medical Image Synthesis](https://mlsanity.com/paper/1908.04346/) (2019)

## Built on by (most cited first, 10 of 10)

- [Taming Vision-Language Models for Medical Image Analysis: A Comprehensive Review](https://mlsanity.com/paper/2506.18378/) (2025)
- [DAMPER: A Dual-Stage Medical Report Generation Framework with Coarse-Grained MeSH Alignment and Fine-Grained Hypergraph Matching](https://mlsanity.com/paper/2412.14535/) (2024)
- [Towards Virtual Clinical Trials of Radiology AI with Conditional Generative Modeling](https://mlsanity.com/paper/2502.09688/) (2025)
- [GEM: Context-Aware Gaze EstiMation with Visual Search Behavior Matching for Chest Radiograph](https://mlsanity.com/paper/2408.05502/) (2024)
- [MeDSLIP: Medical Dual-Stream Language-Image Pre-training with Pathology-Anatomy Semantic Alignment](https://mlsanity.com/paper/2403.10635/) (2024)
- [T-DuMpRa: Teacher-guided Dual-path Multi-prototype Retrieval Augmented framework for fine-grained medical image classification](https://mlsanity.com/paper/2604.17360/) (2026)
- [A Human-Centered Approach to Identifying Promises, Risks, & Challenges of Text-to-Image Generative AI in Radiology](https://mlsanity.com/paper/2507.16207/) (2025)
- [Reducing Domain Gap with Diffusion-Based Domain Adaptation for Cell Counting](https://mlsanity.com/paper/2512.11763/) (2025)
- [Boosting Overlapping Organoid Instance Segmentation Using Pseudo-Label Unmixing and Synthesis-Assisted Learning](https://mlsanity.com/paper/2601.06642/) (2026)
- [Visually-Guided Controllable Medical Image Generation via Fine-Grained Semantic Disentanglement](https://mlsanity.com/paper/2603.10519/) (2026)

## Abstract (arXiv)

Data scarcity and privacy concerns limit the availability of high-quality medical images for public use, which can be mitigated through medical image synthesis. However, current medical image synthesis methods often struggle to accurately capture the complexity of detailed anatomical structures and pathological conditions. To address these challenges, we propose a novel medical image synthesis model that leverages fine-grained image-text alignment and anatomy-pathology prompts to generate highly detailed and accurate synthetic medical images. Our method integrates advanced natural language processing techniques with image generative modeling, enabling precise alignment between descriptive text prompts and the synthesized images' anatomical and pathological details. The proposed approach consists of two key components: an anatomy-pathology prompting module and a fine-grained alignment-based synthesis module. The anatomy-pathology prompting module automatically generates descriptive prompts for high-quality medical images. To further synthesize high-quality medical images from the generated prompts, the fine-grained alignment-based synthesis module pre-defines a visual codebook for the radiology dataset and performs fine-grained alignment between the codebook and generated prompts to obtain key patches as visual clues, facilitating accurate image synthesis. We validate the superiority of our method through experiments on public chest X-ray datasets and demonstrate that our synthetic images preserve accurate semantic information, making them valuable for various medical applications.