mlsanityData to 7 Oct 2026

Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding

arXiv 2205.11487 ↗First posted 23 May 2022cs.CV cs.LGcited by 6,322

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We present Imagen, a text-to-image diffusion model with an unprecedented degree of photorealism and a deep level of language understanding. Imagen builds on the power of large transformer language models in understanding text and hinges on the strength of diffusion models in high-fidelity image generation. Our key discovery is that generic large language models (e.g. T5), pretrained on text-only corpora, are surprisingly effective at encoding text for image synthesis: increasing the size of the language model in Imagen boosts both sample fidelity and image-text alignment much more than increasing the size of the image diffusion model. Imagen achieves a new state-of-the-art FID score of 7.27 on the COCO dataset, without ever training on COCO, and human raters find Imagen samples to be on par with the COCO data itself in image-text alignment. To assess text-to-image models in greater depth, we introduce DrawBench, a comprehensive and challenging benchmark for text-to-image models. With DrawBench, we compare Imagen with recent methods including VQ-GAN+CLIP, Latent Diffusion Models, and DALL-E 2, and find that human raters prefer Imagen over other models in side-by-side comparisons, both in terms of sample quality and image-text alignment. See https://imagen.research.google/ for an overview of the results.

Key facts. Cited by 6,322 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 65 of its 108 references that are in the map. First posted on arXiv 23 May 2022.

Builds on

65 of 108 references.

Closest ancestors · 3

Same direction, newest first

  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  2. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  3. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 62

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  11. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  12. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
38 more
  1. 2016Layer Normalizationcited by 5,579
  2. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  10. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  11. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,506
  13. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  14. 2018Datasheets for Datasetscited by 1,371
  15. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  16. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  17. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  18. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  19. 2021Variational Diffusion Modelscited by 1,145
  20. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,041
  21. 2016Generative Adversarial Text to Image Synthesiscited by 735
  22. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  23. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  24. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  25. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modelingcited by 594
  26. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  27. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
  28. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  29. 2022VQGAN-CLIP: Open Domain Image Generation and Editing with Natural Language Guidancecited by 285
  30. 2020Data and its (dis)contents: A survey of dataset development and use in machine learning researchcited by 259
  31. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  32. 2021Deblurring via Stochastic Refinementcited by 248
  33. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  34. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  35. 2015Deep Generative Image Models using a Laplacian Pyramid of Adversarial Networkscited by 214
  36. 2019Identifying and Reducing Gender Bias in Word-Level Language Modelscited by 214
  37. 2022Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AIcited by 169
  38. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 343; 2023: 1775; 2024: 2008; 2025: 1448; 2026: 748.

6,322 papers.

Sort:

By area:

  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2022Scalable Diffusion Models with Transformerscited by 5,931
  3. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  7. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  8. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  11. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  12. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
38 more
  1. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  2. 2023Consistency Modelscited by 1,778
  3. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,755
  5. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  6. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,483
  7. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  8. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  9. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  10. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  11. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  12. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,285
  13. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  14. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,172
  15. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,168
  16. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  17. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,099
  18. 2022Human Motion Diffusion Modelcited by 1,062
  19. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  20. 2022Diffusion Models in Vision: A Surveycited by 1,008
  21. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  22. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
  23. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  24. 2022CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformerscited by 973
  25. 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
  26. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  27. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  28. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  29. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  30. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  31. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  32. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  33. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 897
  34. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  35. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  36. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 847
  37. 2023DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creationcited by 837
  38. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834

50 most cited of 6,322 All 6,322 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026TextLDM: Language Modeling with Continuous Latent Diffusion
  2. 2025Exploring the Deep Fusion of Large Language Models and Diffusion Transformers for Text-to-Image Synthesis
  3. 2026i1: A Simple and Fully Open Recipe for Strong Text-to-Image Models
  4. 2024Imagen 3
  5. 2025Weak Supervision Dynamic KL-Weighted Diffusion Models Guided by Large Language Models
  6. 2023A Picture is Worth a Thousand Words: Principled Recaptioning Improves Image Generation
  7. 2026High-Fidelity Text-to-Image Generation from Pre-Trained Vision-Language Models via Distribution-Conditioned Diffusion Decoding
  8. 2025Muddit: Liberating Generation Beyond Text-to-Image with a Unified Discrete Diffusion Model