# Text2CT: Towards 3D CT Volume Generation from Free-text Descriptions Using Diffusion Model

arXiv: https://arxiv.org/abs/2505.04522 · HTML: https://mlsanity.com/paper/2505.04522/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Medical imaging and segmentation, one of 14 directions in Diffusion models.
- Builds on 40 of its 70 references that are in the map.
- First posted on arXiv 7 May 2025.

## Placed in

- Diffusion models / [Medical imaging and segmentation](https://mlsanity.com/diffusion/medical-imaging-and-segmentation/)

## Builds on (most cited first, 30 of 40)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Movie Gen: A Cast of Media Foundation Models](https://mlsanity.com/paper/2410.13720/) (2024)
- [TotalSegmentator: robust segmentation of 104 anatomical structures in CT images](https://mlsanity.com/paper/2208.05868/) (2022)
- [BiomedCLIP: a multimodal biomedical foundation model pretrained from fifteen million scientific image-text pairs](https://mlsanity.com/paper/2303.00915/) (2023)
- [Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors](https://mlsanity.com/paper/2203.13131/) (2022)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [Diffusion Models for Medical Image Analysis: A Comprehensive Survey](https://mlsanity.com/paper/2211.07804/) (2022)
- [Text-to-image Diffusion Models in Generative AI: A Survey](https://mlsanity.com/paper/2303.07909/) (2023)
- [Controllable Text-to-Image Generation](https://mlsanity.com/paper/1909.07083/) (2019)
- [MirrorGAN: Learning Text-to-image Generation by Redescription](https://mlsanity.com/paper/1903.05854/) (2019)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Transformer for Single Image Super-Resolution](https://mlsanity.com/paper/2108.11084/) (2021)
- [Medical Image Synthesis for Data Augmentation and Anonymization using Generative Adversarial Networks](https://mlsanity.com/paper/1807.10225/) (2018)
- [Deep Learning for Medical Image Processing: Overview, Challenges and Future](https://mlsanity.com/paper/1704.06825/) (2017)
- [MAISI: Medical AI for Synthetic Imaging](https://mlsanity.com/paper/2409.11169/) (2024)
- [Multi-institutional Collaborations for Improving Deep Learning-based Magnetic Resonance Image Reconstruction Using Federated Learning](https://mlsanity.com/paper/2103.02148/) (2021)
- [3D Deep Learning on Medical Images: A Review](https://mlsanity.com/paper/2004.00218/) (2020)
- [The Creativity of Text-to-Image Generation](https://mlsanity.com/paper/2206.02904/) (2022)
- [Machine-Learning-Based Multiple Abnormality Prediction with Large-Scale Chest Computed Tomography Volumes](https://mlsanity.com/paper/2002.04752/) (2020)
- [Real-World Single Image Super-Resolution: A Brief Review](https://mlsanity.com/paper/2103.02368/) (2021)

## Built on by (most cited first, 13 of 13)

- [Scaling Tumor Segmentation: Best Lessons from Real and Synthetic Data](https://mlsanity.com/paper/2510.14831/) (2025)
- [Radiology Report Conditional 3D CT Generation with Multi Encoder Latent diffusion Model](https://mlsanity.com/paper/2509.14780/) (2025)
- [See More, Change Less: Anatomy-Aware Diffusion for Contrast Enhancement](https://mlsanity.com/paper/2512.07251/) (2025)
- [Exploiting Completeness Perception with Diffusion Transformer for Unified 3D MRI Synthesis](https://mlsanity.com/paper/2602.18400/) (2026)
- [Any2Any 3D Diffusion Models with Knowledge Transfer: A Radiotherapy Planning Study](https://mlsanity.com/paper/2605.09622/) (2026)
- [Foundation VAEs for 3D CT Reconstruction, Augmentation, and Generation](https://mlsanity.com/paper/2605.30893/) (2026)
- [Knowledge-Guided 3D CT Generation: A Conditioning-Centric Taxonomy](https://mlsanity.com/paper/2608.09992/) (2026)
- [ShapeKit](https://mlsanity.com/paper/2506.24003/) (2025)
- [Imagining Alternatives: Towards High-Resolution 3D Counterfactual Medical Image Generation via Language Guidance](https://mlsanity.com/paper/2509.05978/) (2025)
- [CSEval: A Framework for Evaluating Clinical Semantics in Text-to-Image Generation](https://mlsanity.com/paper/2602.12004/) (2026)
- [PHASOR: Anatomy- and Phase-Consistent Volumetric Diffusion for CT Virtual Contrast Enhancement](https://mlsanity.com/paper/2604.01053/) (2026)
- [Distilling Photon-Counting CT into Routine Chest CT through Clinically Validated Degradation Modeling](https://mlsanity.com/paper/2604.07329/) (2026)
- [MedSyn2: Flexible Control of 3D CT Generation via Text and Semantically-Defined Segmentation Prompts](https://mlsanity.com/paper/2606.00967/) (2026)

## Abstract (arXiv)

Generating 3D CT volumes from descriptive free-text inputs presents a transformative opportunity in diagnostics and research. In this paper, we introduce Text2CT, a novel approach for synthesizing 3D CT volumes from textual descriptions using the diffusion model. Unlike previous methods that rely on fixed-format text input, Text2CT employs a novel prompt formulation that enables generation from diverse, free-text descriptions. The proposed framework encodes medical text into latent representations and decodes them into high-resolution 3D CT scans, effectively bridging the gap between semantic text inputs and detailed volumetric representations in a unified 3D framework. Our method demonstrates superior performance in preserving anatomical fidelity and capturing intricate structures as described in the input text. Extensive evaluations show that our approach achieves state-of-the-art results, offering promising potential applications in diagnostics, and data augmentation.