mlsanityData to 14 Oct 2026

GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models

arXiv 2112.10741 ↗First posted 20 Dec 2021cs.CV cs.GR cs.LGcited by 3,614

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Diffusion models have recently been shown to generate high-quality synthetic images, especially when paired with a guidance technique to trade off diversity for fidelity. We explore diffusion models for the problem of text-conditional image synthesis and compare two different guidance strategies: CLIP guidance and classifier-free guidance. We find that the latter is preferred by human evaluators for both photorealism and caption similarity, and often produces photorealistic samples. Samples from a 3.5 billion parameter text-conditional diffusion model using classifier-free guidance are favored by human evaluators to those from DALL-E, even when the latter uses expensive CLIP reranking. Additionally, we find that our models can be fine-tuned to perform image inpainting, enabling powerful text-driven image editing. We train a smaller model on a filtered dataset and release the code and weights at https://github.com/openai/glide-text2im.

Key facts. Cited by 3,614 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 35 of its 51 references that are in the map. First posted on arXiv 20 Dec 2021.

Builds on

35 of 51 references.

Closest ancestors · 2

Same direction, newest first

  1. 2021Blended Diffusion for Text-driven Editing of Natural Images29 Nov
  2. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,122
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  9. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  10. 2017Neural Discrete Representation Learningcited by 4,712
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  12. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
21 more
  1. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  2. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  3. 2016Improved Techniques for Training GANscited by 2,970
  4. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  5. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2017Mixed Precision Trainingcited by 1,156
  9. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  10. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  11. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 944
  12. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 774
  13. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  14. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  15. 2021Cross-Modal Contrastive Learning for Text-to-Image Generationcited by 259
  16. 2021LAFITE: Towards Language-Free Training for Text-to-Image Generationcited by 128
  17. 2021Paint by Wordcited by 107
  18. 2021Improving Text-to-Image Synthesis Using Contrastive Learningcited by 62
  19. 2019HYPE: A Benchmark for Human eYe Perceptual Evaluation of Generative Modelscited by 59
  20. 2021Generating images from caption and vice versa via CLIP-Guided Generative Latent Space Searchcited by 57
  21. 2020Conditional Image Generation and Manipulation for User-Specified Contentcited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 2; 2022: 290; 2023: 1098; 2024: 1133; 2025: 746; 2026: 345.

3,614 papers.

Sort:

By area:

  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  2. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  3. 2022Scalable Diffusion Models with Transformerscited by 6,054
  4. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,503
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  6. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  9. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
38 more
  1. 2022Video Diffusion Modelscited by 2,305
  2. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  3. 2023Consistency Modelscited by 1,822
  4. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  6. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  7. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  8. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  9. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  10. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  11. 2022RePaint: Inpainting using Denoising Diffusion Probabilistic Modelscited by 1,430
  12. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  13. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  14. 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
  15. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  16. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  17. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  18. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  19. 2022Human Motion Diffusion Modelcited by 1,080
  20. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  21. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  22. 2023One-step Diffusion with Distribution Matching Distillationcited by 1,019
  23. 2022Diffusion Models in Vision: A Surveycited by 1,015
  24. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  25. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 961
  26. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  27. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  28. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 929
  29. 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
  30. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  31. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 846
  32. 2023GLIGEN: Open-Set Grounded Text-to-Image Generationcited by 781
  33. 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 780
  34. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  35. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  36. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  37. 2023Extracting Training Data from Diffusion Modelscited by 732
  38. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727

50 most cited of 3,614 All 3,614 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023Perceptual Similarity guidance and text guidance optimization for Editing Real Images using Guided Diffusion Models
  2. 2021More Control for Free! Image Synthesis with Semantic Diffusion Guidance
  3. 2024Diffusion-Based Conditional Image Editing through Optimized Inference with Guidance
  4. 2022Direct Inversion: Optimization-Free Text-Driven Real Image Editing with Diffusion Models
  5. 2023Custom-Edit: Text-Guided Image Editing with Customized Diffusion Models
  6. 2023EditVal: Benchmarking Diffusion Based Text-Guided Image Editing Methods
  7. 2023Design Booster: A Text-Guided Diffusion Model for Image Translation with Spatial Layout Preservation
  8. 2023Filter-Guided Diffusion for Controllable Image Generation