# What Does DALL-E 2 Know About Radiology?

arXiv: https://arxiv.org/abs/2209.13696 · HTML: https://mlsanity.com/paper/2209.13696/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 19 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 13 of 13)

- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [A very preliminary analysis of DALL-E 2](https://mlsanity.com/paper/2204.13807/) (2022)
- [The Creativity of Text-to-Image Generation](https://mlsanity.com/paper/2206.02904/) (2022)
- [Synthetic Data -- what, why and how?](https://mlsanity.com/paper/2205.03257/) (2022)
- [Testing Relational Understanding in Text-Guided Image Generation](https://mlsanity.com/paper/2208.00005/) (2022)
- [Overcoming Barriers to Data Sharing with Medical Image Generation: A Comprehensive Evaluation](https://mlsanity.com/paper/2012.03769/) (2020)
- [Synthetic Data: Opening the data floodgates to enable faster, more directed development of machine learning methods](https://mlsanity.com/paper/2012.04580/) (2020)
- [A multi-stage GAN for multi-organ chest X-ray image generation and segmentation](https://mlsanity.com/paper/2106.05132/) (2021)

## Built on by (most cited first, 7 of 7)

- [Aligning Synthetic Medical Images with Clinical Knowledge using Human Feedback](https://mlsanity.com/paper/2306.12438/) (2023)
- [Prompt to Polyp: Medical Text-Conditioned Image Synthesis with Diffusion Models](https://mlsanity.com/paper/2505.05573/) (2025)
- [Data Can Speak for Itself: Quality-guided Utilization of Wireless Synthetic Data](https://mlsanity.com/paper/2506.23174/) (2025)
- [A Misleading Gallery of Fluid Motion by Generative Artificial Intelligence](https://mlsanity.com/paper/2405.15406/) (2024)
- [Spatio-Temporal Analysis of Patient-Derived Organoid Videos Using Deep Learning for the Prediction of Drug Efficacy](https://mlsanity.com/paper/2308.14461/) (2023)
- [Guardrails for avoiding harmful medical product recommendations and off-label promotion in generative AI models](https://mlsanity.com/paper/2406.16455/) (2024)
- [Hallucination Behavior in Multimodal LLMs Across Agricultural Image Interpretation and Generation Tasks](https://mlsanity.com/paper/2605.27595/) (2026)

## Abstract (arXiv)

Generative models such as DALL-E 2 could represent a promising future tool for image generation, augmentation, and manipulation for artificial intelligence research in radiology provided that these models have sufficient medical domain knowledge. Here we show that DALL-E 2 has learned relevant representations of X-ray images with promising capabilities in terms of zero-shot text-to-image generation of new images, continuation of an image beyond its original boundaries, or removal of elements, while pathology generation or CT, MRI, and ultrasound images are still limited. The use of generative models for augmenting and generating radiological data thus seems feasible, even if further fine-tuning and adaptation of these models to the respective domain is required beforehand.