mlsanityData to 14 Oct 2026

Best Prompts for Text-to-Image Models and How to Find Them

arXiv 2209.11711 ↗First posted 23 Sep 2022cs.HC cs.CL cs.CVcited by 35

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Recent progress in generative models, especially in text-guided diffusion models, has enabled the production of aesthetically-pleasing imagery resembling the works of professional human artists. However, one has to carefully compose the textual description, called the prompt, and augment it with a set of clarifying keywords. Since aesthetics are challenging to evaluate computationally, human feedback is needed to determine the optimal prompt formulation and keyword combination. In this paper, we present a human-in-the-loop approach to learning the most useful combination of prompt keywords using a genetic algorithm. We also show how such an approach can improve the aesthetic appeal of images depicting the same descriptions.

Key facts. Cited by 35 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 17 of its 24 references that are in the map. First posted on arXiv 23 Sep 2022.

Builds on

17 of 24 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 16

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  6. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  7. 2020Making Pre-trained Language Models Better Few-shot Learnerscited by 1,256
  8. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,135
  9. 2021Cross-Task Generalization via Natural Language Crowdsourcing Instructionscited by 699
  10. 2021Learning To Retrieve Prompts for In-Context Learningcited by 648
  11. 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 606
  12. 2021Design Guidelines for Prompt Engineering Text-to-Image Generative Modelscited by 218
4 more
  1. 2020How Context Affects Language Models' Factual Predictionscited by 181
  2. 2021Learning from Crowds with Crowd-Kitcited by 24
  3. 2021Image Composition Assessment with Saliency-augmented Multi-pattern Poolingcited by 21
  4. 2021IMDB-WIKI-SbS: An Evaluation Dataset for Crowdsourced Pairwise Comparisonscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 9; 2024: 17; 2025: 6; 2026: 2.

35 papers.

Sort:

By area:

  1. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  2. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
  3. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
  4. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 62
  5. 2023Prompt Stealing Attacks Against Text-to-Image Generation Modelscited by 51
  6. 2024Coarse-to-Fine Latent Diffusion for Pose-Guided Person Image Synthesiscited by 48
  7. 2024Dynamic Prompt Optimizing for Text-to-Image Generationcited by 45
  8. 2024Diffusion Facial Forgery Detectioncited by 41
  9. 2023A Prompt Log Analysis of Text-to-Image Generation Systemscited by 24
  10. 2024Q-Refine: A Perceptual Quality Refiner for AI-Generated Imagecited by 21
  11. 2024Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generationcited by 20
  12. 2024Language-Driven Interactive Traffic Trajectory Generationcited by 19
23 more
  1. 2023Text-to-Image Generation for Abstract Conceptscited by 18
  2. 2023BeautifulPrompt: Towards Automatic Prompt Engineering for Text-to-Image Synthesiscited by 18
  3. 2023Diffusion Explainer: Visual Explanation for Text-to-image Stable Diffusioncited by 17
  4. 2024Generating Compositional Scenes via Text-to-image RGBA Instance Generationcited by 14
  5. 2024Prompt Optimizer of Text-to-Image Diffusion Models for Abstract Concept Understandingcited by 9
  6. 2024Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesiscited by 9
  7. 2023Vision + Language Applications: A Surveycited by 8
  8. 2025PromptMap: An Alternative Interaction Style for AI-Based Image Generationcited by 8
  9. 2024No Longer Trending on Artstation: Prompt Analysis of Generative AI Artcited by 7
  10. 2025VisualPrompter: Semantic-Aware Prompt Optimization with Visual Feedback for Text-to-Image Synthesiscited by 6
  11. 2024SCAPE: Searching Conceptual Architecture Prompts using Evolutioncited by 4
  12. 2024CoProNN: Concept-based Prototypical Nearest Neighbors for Explaining Vision Modelscited by 2
  13. 2024PQPP: A Joint Benchmark for Text-to-Image Prompt and Query Performance Predictioncited by 2
  14. 2025Grounding Text-to-Image Diffusion Models for Controlled High-Quality Image Generationcited by 2
  15. 2024Human Aesthetic Preference-Based Large Text-to-Image Model Personalization: Kandinsky Generation as an Examplecited by 1
  16. 2025MineTheGap: Automatic Mining of Biases in Text-to-Image Modelscited by 1
  17. 2026One Rewrite to Fix Them All? Type-Aware Repair Allocation for Text-to-Image Prompt Optimizationcited by 1
  18. 2023Exploring the Effectiveness of Dataset Synthesis: An application of Apple Detection in Orchardscited by 0
  19. 2024SSP: A Simple and Safe automatic Prompt engineering method towards realistic image synthesis on LVMcited by 0
  20. 2024Machine Apophenia: The Kaleidoscopic Generation of Architectural Imagescited by 0
  21. 2025CLEAR: Cue Learning using Evolution for Accurate Recognition Applied to Sustainability Data Extractioncited by 0
  22. 2025AdaptaGen: Domain-Specific Image Generation through Hierarchical Semantic Optimization Frameworkcited by 0
  23. 2026DOGS: Design-Space Sampling for Prompt-Driven Logo Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Evolutionary Token-Level Prompt Optimization for Diffusion Models
  2. 2022Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models
  3. 2022Optimizing Prompts for Text-to-Image Generation
  4. 2025Self-Rewarding Large Vision-Language Models for Optimizing Prompts in Text-to-Image Generation
  5. 2023NeuroPrompts: An Adaptive Framework to Optimize Prompts for Text-to-Image Generation
  6. 2022Language Does More Than Describe: On The Lack Of Figurative Speech in Text-To-Image Models
  7. 2023Manipulating Embeddings of Stable Diffusion Prompts
  8. 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation