mlsanityData to 14 Oct 2026

Diffusion-Based Visual Art Creation: A Survey and New Perspectives

arXiv 2408.12128 ↗First posted 22 Aug 2024cs.AI cs.CVcited by 17

Not on the map. Why?

Abstract (arXiv)

The integration of generative AI in visual art has revolutionized not only how visual content is created but also how AI interacts with and reflects the underlying domain knowledge. This survey explores the emerging realm of diffusion-based visual art creation, examining its development from both artistic and technical perspectives. We structure the survey into three phases, data feature and framework identification, detailed analyses using a structured coding process, and open-ended prospective outlooks. Our findings reveal how artistic requirements are transformed into technical challenges and highlight the design and application of diffusion-based methods within visual art creation. We also provide insights into future directions from technical and synergistic perspectives, suggesting that the confluence of generative AI and art has shifted the creative paradigm and opened up new possibilities. By summarizing the development and trends of this emerging interdisciplinary area, we aim to shed light on the mechanisms through which AI systems emulate and possibly, enhance human capacities in artistic perception and creativity.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 117 of its 194 references that are in the map. First posted on arXiv 22 Aug 2024.

Builds on

117 of 194 references.

References · 117

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2020Denoising Diffusion Implicit Modelscited by 9,781
  7. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2017A Unified Approach to Interpreting Model Predictionscited by 6,555
  10. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
38 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  3. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  4. 2016Improved Techniques for Training GANscited by 2,970
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  6. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  7. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
  8. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
  9. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  10. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  11. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  12. 2022Imagic: Text-Based Real Image Editing with Diffusion Modelscited by 1,178
  13. 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
  14. 2021StyleCLIP: Text-Driven Manipulation of StyleGAN Imagerycited by 991
  15. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 931
  16. 2018Self-Attention Generative Adversarial Networkscited by 809
  17. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
  18. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  19. 2023Muse: Text-To-Image Generation via Masked Generative Transformerscited by 648
  20. 2021ILVR: Conditioning Method for Denoising Diffusion Probabilistic Modelscited by 635
  21. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  22. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  23. 2023Instruct-NeRF2NeRF: Editing 3D Scenes with Instructionscited by 484
  24. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  25. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  26. 2021GAN Inversion: A Surveycited by 367
  27. 2024Retrieval-Augmented Generation for AI-Generated Content: A Surveycited by 354
  28. 2022Inversion-Based Style Transfer with Diffusion Modelscited by 291
  29. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 233
  30. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  31. 2023A Comprehensive Survey of AI-Generated Content (AIGC): A History of Generative AI from GAN to ChatGPTcited by 224
  32. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  33. 2017Neural Style Transfer: A Reviewcited by 199
  34. 2023ZipLoRA: Any Subject in Any Style by Effectively Merging LoRAscited by 194
  35. 2023Adversarial Example Does Good: Preventing Painting Imitation from Diffusion Models via Adversarial Examplescited by 167
  36. 2023WonderJourney: Going from Anywhere to Everywherecited by 158
  37. 2023ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Modelscited by 128
  38. 2024Transparent Image Layer Diffusion using Latent Transparencycited by 114

50 most cited of 117 All 117 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 11; 2026: 4.

17 papers.

Sort:

By area:

  1. 2025Enhancing Creative Generation on Stable Diffusion-based Modelscited by 13
  2. 2025Follow-Your-Color: Multi-Instance Sketch Colorizationcited by 10
  3. 2025Evaluating and Predicting Distorted Human Body Parts for Generated Imagescited by 5
  4. 2025Neural-Driven Image Editingcited by 4
  5. 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Modelscited by 3
  6. 2024ILDiff: Generate Transparent Animated Stickers by Implicit Layout Distillationcited by 2
  7. 2025PT-Mark: Invisible Watermarking for Text-to-image Diffusion Models via Semantic-aware Pivotal Tuningcited by 2
  8. 2025Backdoor Defense in Diffusion Models via Spatial Attention Unlearningcited by 2
  9. 2026GRRE: Leveraging G-Channel Removed Reconstruction Error for Robust Detection of AI-Generated Imagescited by 1
  10. 2026HanMoVLM: Large Vision-Language Models for Professional Artistic Painting Evaluationcited by 1
  11. 2024MagicStyle: Portrait Stylization Based on Reference Imagecited by 0
  12. 2025MagicAnime: A Hierarchically Annotated, Multimodal and Multitasking Dataset with Benchmarks for Cartoon Animation Generationcited by 0
5 more
  1. 2025Attacks on Approximate Caches in Text-to-Image Diffusion Modelscited by 0
  2. 2025PC-Diffusion: Aligning Diffusion Models with Human Preferences via Preference Classifiercited by 0
  3. 2025GreenPlanner: Practical Floorplan Layout Generation via an Energy-Aware and Function-Feasible Generative Frameworkcited by 0
  4. 2026Attention Dynamics in Diffusion Models: A Visual Analytics Framework for Human-AI Collaborationcited by 0
  5. 2026UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generationcited by 0