mlsanityData to 7 Oct 2026

A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions

arXiv 2308.13142 ↗First posted 25 Aug 2023cs.CV cs.AIcited by 23

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Recently, there has been significant progress in the development of large models. Following the success of ChatGPT, numerous language models have been introduced, demonstrating remarkable performance. Similar advancements have also been observed in image generation models, such as Google's Imagen model, OpenAI's DALL-E 2, and stable diffusion models, which have exhibited impressive capabilities in generating images. However, similar to large language models, these models still encounter unresolved challenges. Fortunately, the availability of open-source stable diffusion models and their underlying mathematical principles has enabled the academic community to extensively analyze the performance of current image generation models and make improvements based on this stable diffusion framework. This survey aims to examine the existing issues and the current solutions pertaining to image generation models.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 73 of its 83 references that are in the map. First posted on arXiv 25 Aug 2023.

Builds on

73 of 83 references.

Closest ancestors · 27

Same direction, newest first

  1. 2023DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing26 Jun
  2. 2023Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment15 Jun
  3. 2023Grounded Text-to-Image Synthesis with Attention Refocusing8 Jun
  4. 2023Detector Guidance for Multi-Object Text-to-Image Generation4 Jun
  5. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models25 May
  6. 2023Compositional Text-to-Image Synthesis with Attention Map Control of Diffusion Models23 May
  7. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models23 May
  8. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wild18 May
  9. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention17 May
  10. 2023GlueGen: Plug and Play Multi-modal Encoders for X-to-image Generation17 Mar
  11. 2023Text-to-image Diffusion Models in Generative AI: A Survey14 Mar
  12. 2023Encoder-based Domain Tuning for Fast Personalization of Text-to-Image Models23 Feb
15 more
  1. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models16 Feb
  2. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  3. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  4. 2023GLIGEN: Open-Set Grounded Text-to-Image Generation17 Jan
  5. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  6. 2022Multi-Concept Customization of Text-to-Image Diffusion8 Dec
  7. 2022SpaText: Spatio-Textual Representation for Controllable Image Generation25 Nov
  8. 2022Shifted Diffusion for Text-to-image Generation24 Nov
  9. 2022InstructPix2Pix: Learning to Follow Image Editing Instructions17 Nov
  10. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  11. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Experts27 Oct
  12. 2022Imagic: Text-Based Real Image Editing with Diffusion Models17 Oct
  13. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation25 Aug
  14. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  15. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 46

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2022Training language models to follow instructions with human feedbackcited by 15,712
  8. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
34 more
  1. 2022Classifier-Free Diffusion Guidancecited by 5,717
  2. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  3. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,413
  4. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  5. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  8. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,612
  9. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  10. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  11. 2022Diffusion Models in Vision: A Surveycited by 1,008
  12. 2023Training Diffusion Models with Reinforcement Learningcited by 939
  13. 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 603
  14. 2022All are Worth Words: A ViT Backbone for Diffusion Modelscited by 498
  15. 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 438
  16. 2023Universal Guidance for Diffusion Modelscited by 426
  17. 2023Aligning Text-to-Image Models using Human Feedbackcited by 418
  18. 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 414
  19. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  20. 2023Training-Free Layout Control with Cross-Attention Guidancecited by 312
  21. 2021An Empirical Study of Training End-to-End Vision-and-Language Transformerscited by 271
  22. 2023Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifoldcited by 218
  23. 2022Vision-Language Pre-Training with Triple Contrastive Learningcited by 212
  24. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  25. 2022Optimizing Prompts for Text-to-Image Generationcited by 204
  26. 2022ReCo: Region-Controlled Text-to-Image Generationcited by 177
  27. 2022Improving Sample Quality of Diffusion Models Using Self-Attention Guidancecited by 151
  28. 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 135
  29. 2023Generative Diffusion Models on Graphs: Methods and Applicationscited by 79
  30. 2023A Survey on Audio Diffusion Models: Text To Speech Synthesis and Enhancement in Generative AIcited by 68
  31. 2023Visual Programming for Text-to-Image Generation and Evaluationcited by 53
  32. 2022SceneComposer: Any-Level Semantic Image Synthesiscited by 43
  33. 2022Semi-Parametric Neural Image Synthesiscited by 36
  34. 2023Training-Free Location-Aware Text-to-Image Synthesiscited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 12; 2025: 8; 2026: 3.

23 papers.

Sort:

By area:

  1. 2024Diffusion Model-Based Image Editing: A Surveycited by 212
  2. 2024ConceptPrune: Concept Editing in Diffusion Models via Skilled Neuron Pruningcited by 51
  3. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  4. 2024Diffusion Model for Planning: A Systematic Literature Reviewcited by 17
  5. 2024Generalizing Alignment Paradigm of Text-to-Image Generation with Preferences through f-divergence Minimizationcited by 15
  6. 2025From Image to Video: An Empirical Study of Diffusion Representationscited by 14
  7. 2024What You See Is What Matters: A Novel Visual and Physics-Based Metric for Evaluating Video Generation Qualitycited by 6
  8. 2024Physics-Inspired Generative Models in Medical Imaging: A Reviewcited by 5
  9. 2024Zero-Shot Distillation for Image Encoders: How to Make Effective Use of Synthetic Datacited by 4
  10. 2024The Fabrication of Reality and Fantasy: Scene Generation with LLM-Assisted Prompt Interpretationcited by 2
  11. 2024Toward Scene Graph and Layout Guided Complex 3D Scene Generationcited by 2
  12. 2025TextDiffSeg: Text-guided Latent Diffusion Model for 3d Medical Images Segmentationcited by 2
11 more
  1. 2025REDEditing: Relationship-Driven Precise Backdoor Poisoning on Text-to-Image Diffusion Modelscited by 2
  2. 2025FeatInv: Spatially resolved mapping from feature space to input space using conditional diffusion modelscited by 2
  3. 2024FacEnhance: Facial Expression Enhancing with Recurrent DDPMscited by 1
  4. 2025TextTIGER: Text-based Intelligent Generation with Entity Prompt Refinement for Text-to-Image Generationcited by 1
  5. 2025Range-Edit: Semantic Mask Guided Outdoor LiDAR Scene Editingcited by 1
  6. 2026AMVICC: A Novel Benchmark for Cross-Modal Failure Mode Profiling for VLMs and IGMscited by 1
  7. 2024Track Anything Behind Everything: Zero-Shot Amodal Video Object Segmentationcited by 0
  8. 2025ASMR: Augmenting Life Scenario using Large Generative Models for Robotic Action Reflectioncited by 0
  9. 2025Detecting AI-Generated Images via Diffusion Snap-Back Reconstruction: A Forensic Approachcited by 0
  10. 2026Polaris: Scaling Up Instruction-Guided Image Generation Towards Millions of Personalized Style Needscited by 0
  11. 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023State of the Art on Diffusion Models for Visual Computing
  2. 2024Controllable Generation with Text-to-Image Diffusion Models: A Survey
  3. 2022Implementing and Experimenting with Diffusion Models for Text-to-Image Generation
  4. 2023Nested Diffusion Processes for Anytime Image Generation
  5. 2024Generative AI in Vision: A Survey on Models, Metrics and Applications
  6. 2025Efficient Diffusion Models: A Survey
  7. 2023A Survey on Video Diffusion Models
  8. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices