PromptEvolver: Prompt Inversion through Evolutionary Optimization in Natural-Language Space
Not on the map. Why?
Abstract (arXiv)
Text-to-image generation has progressed rapidly, but faithfully generating complex scenes requires extensive trial-and-error to find the exact prompt. In the prompt inversion task, the goal is to recover a textual prompt that can faithfully reconstruct a given target image. Currently, existing methods frequently yield suboptimal reconstructions and produce unnatural, hard-to-interpret prompts that hinder transparency and controllability. In this work, we present PromptEvolver, a prompt inversion approach that generates natural-language prompts while achieving high-fidelity reconstructions of the target image. Our method uses a genetic algorithm to optimize the prompt, leveraging a strong vision-language model to guide the evolution process. Importantly, it works on black-box generation models by requiring only image outputs. Finally, we evaluate PromptEvolver across multiple prompt inversion benchmarks and show that it consistently outperforms competing methods.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 42 of its 55 references that are in the map. First posted on arXiv 3 Apr 2026.
Builds on
42 of 55 references.
References · 42
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
- 2014Deep Learning Face Attributes in the Wildcited by 3,693
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
30 more
- 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
- 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
- 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
- 2022Multi-Concept Customization of Text-to-Image Diffusioncited by 1,106
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2022Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translationcited by 906
- 2023Adversarial Diffusion Distillationcited by 759
- 2022When and why vision-language models behave like bags-of-words, and what to do about it?cited by 610
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
- 2023ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generationcited by 416
- 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
- 2023DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Datacited by 337
- 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 336
- 2022EDICT: Exact Diffusion Inversion via Coupled Transformationscited by 258
- 2023P+: Extended Textual Conditioning in Text-to-Image Generationcited by 242
- 2023LEDITS++: Limitless Image Editing using Text-to-Image Modelscited by 148
- 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 138
- 2024ReNoise: Real Image Inversion Through Iterative Noisingcited by 128
- 2023A Neural Space-Time Representation for Text-to-Image Personalizationcited by 123
- 2025Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversitycited by 105
- 2023Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-To-Image Modelscited by 85
- 2023Effective Real Image Editing with Accelerated Iterative Diffusion Inversioncited by 79
- 2023Prompt Stealing Attacks Against Text-to-Image Generation Modelscited by 51
- 2024Dynamic Prompt Optimizing for Text-to-Image Generationcited by 45
- 2023Prompting Hard or Hardly Prompting: Prompt Inversion for Text-to-Image Diffusion Modelscited by 40
- 2024SPRIG: Improving Large Language Model Performance by System Prompt Optimizationcited by 29
- 2023What is in a Text-to-Image Prompt: The Potential of Stable Diffusion in Visual Arts Educationcited by 14
- 2024Automated Black-box Prompt Engineering for Personalized Text-to-Image Generationcited by 12
- 2025Visually Guided Decoding: Gradient-Free Hard Prompt Inversion with Language Modelscited by 10
- 2023Reverse Stable Diffusion: What prompt was used to generate this image?cited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Evolutionary Token-Level Prompt Optimization for Diffusion Models
- 2025Evolve to Inspire: Novelty Search for Diverse Image Generation
- 2026RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignment
- 2026Dual Inversion for Text-to-Image Diffusion Models: From Both Prompt and Noise Perspectives
- 2024Batch-Instructed Gradient for Prompt Evolution:Systematic Prompt Optimization for Enhanced Text-to-Image Synthesis
- 2025Diversifying Long Prompt Image Generation through Structured Prompt Embedding Space Sampling
- 2022Optimizing Prompts for Text-to-Image Generation
- 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generation