# PromptSculptor: Multi-Agent Based Text-to-Image Prompt Optimization

arXiv: https://arxiv.org/abs/2509.12446 · HTML: https://mlsanity.com/paper/2509.12446/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 59 references that are in the map.
- First posted on arXiv 15 Sep 2025.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [StyleCLIP: Text-Driven Manipulation of StyleGAN Imagery](https://mlsanity.com/paper/2103.17249/) (2021)
- [Automatic Prompt Optimization with "Gradient Descent" and Beam Search](https://mlsanity.com/paper/2305.03495/) (2023)
- [Promptbreeder: Self-Referential Self-Improvement Via Prompt Evolution](https://mlsanity.com/paper/2309.16797/) (2023)
- [Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis](https://mlsanity.com/paper/2212.05032/) (2022)
- [Diffusion Self-Guidance for Controllable Image Generation](https://mlsanity.com/paper/2306.00986/) (2023)
- [Human Preference Score: Better Aligning Text-to-Image Models with Human Preference](https://mlsanity.com/paper/2303.14420/) (2023)
- [Text-to-image Diffusion Models in Generative AI: A Survey](https://mlsanity.com/paper/2303.07909/) (2023)
- [LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models](https://mlsanity.com/paper/2305.13655/) (2023)
- [PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimization](https://mlsanity.com/paper/2310.16427/) (2023)
- [FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving](https://mlsanity.com/paper/2505.17685/) (2025)
- [Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step](https://mlsanity.com/paper/2501.13926/) (2025)
- [Improving Text-to-Image Consistency via Automatic Prompt Optimization](https://mlsanity.com/paper/2403.17804/) (2024)
- [Black-box Prompt Learning for Pre-trained Language Models](https://mlsanity.com/paper/2201.08531/) (2022)
- [PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation](https://mlsanity.com/paper/2307.09036/) (2023)
- [DreamSync: Aligning Text-to-Image Generation with Image Understanding Feedback](https://mlsanity.com/paper/2311.17946/) (2023)
- [Toward Human Readable Prompt Tuning: Kubrick's The Shining is a good movie, and a good prompt too?](https://mlsanity.com/paper/2212.10539/) (2022)
- [If at First You Don't Succeed, Try, Try Again: Faithful Diffusion-based Text-to-Image Generation by Selection](https://mlsanity.com/paper/2305.13308/) (2023)
- [Use Your INSTINCT: INSTruction optimization for LLMs usIng Neural bandits Coupled with Transformers](https://mlsanity.com/paper/2310.02905/) (2023)
- [Bridging the Editing Gap in LLMs: FineEdit for Precise and Targeted Text Modifications](https://mlsanity.com/paper/2502.13358/) (2025)
- [Selective KV-Cache Sharing to Mitigate Timing Side-Channels in LLM Inference](https://mlsanity.com/paper/2508.08438/) (2025)
- [SLIC: Self-Supervised Learning with Iterative Clustering for Human Action Videos](https://mlsanity.com/paper/2206.12534/) (2022)
- [Revisiting OPRO: The Limitations of Small-Scale LLMs as Optimizers](https://mlsanity.com/paper/2405.10276/) (2024)
- [From LLM-anation to LLM-orchestrator: Coordinating Small Models for Data Labeling](https://mlsanity.com/paper/2506.16393/) (2025)
- [Learning Explainable Stock Predictions with Tweets Using Mixture of Experts](https://mlsanity.com/paper/2507.20535/) (2025)

## Built on by (most cited first, 19 of 19)

- [JanusVLN: Decoupling Semantics and Spatiality with Dual Implicit Memory for Vision-Language Navigation](https://mlsanity.com/paper/2509.22548/) (2025)
- [Think with 3D: Geometric Imagination Grounded Spatial Reasoning from Limited Views](https://mlsanity.com/paper/2510.18632/) (2025)
- [HUD: Hierarchical Uncertainty-Aware Disambiguation Network for Composed Video Retrieval](https://mlsanity.com/paper/2512.02792/) (2025)
- [Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation](https://mlsanity.com/paper/2602.01756/) (2026)
- [Think-Then-Generate: Reasoning-Aware Text-to-Image Diffusion with LLM Encoders](https://mlsanity.com/paper/2601.10332/) (2026)
- [SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation](https://mlsanity.com/paper/2605.08043/) (2026)
- [RoboRouter: Training-Free Policy Routing for Robotic Manipulation](https://mlsanity.com/paper/2603.07892/) (2026)
- [AnimeAgent: Is the Multi-Agent via Image-to-Video models a Good Disney Storytelling Artist?](https://mlsanity.com/paper/2602.20664/) (2026)
- [GenRouter: Unified Workflow Routing for Agentic Image Generation](https://mlsanity.com/paper/2608.16721/) (2026)
- [HyperEdit: Unlocking Instruction-based Text Editing in LLMs via Hypernetworks](https://mlsanity.com/paper/2512.12544/) (2025)
- [BrandFusion: A Multi-Agent Framework for Seamless Brand Integration in Text-to-Video Generation](https://mlsanity.com/paper/2603.02816/) (2026)
- [MUSE: Resolving Manifold Misalignment in Visual Tokenization via Topological Orthogonality](https://mlsanity.com/paper/2605.05646/) (2026)
- [SIDiffAgent: Self-Improving Diffusion Agent](https://mlsanity.com/paper/2602.02051/) (2026)
- [PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation](https://mlsanity.com/paper/2607.24353/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [One Tool Is Enough: Reinforcement Learning for Repository-Level LLM Agents](https://mlsanity.com/paper/2512.20957/) (2025)
- [Position: Vector Prompt Interfaces Should Be Exposed to Enable Customization of Large Language Models](https://mlsanity.com/paper/2603.04292/) (2026)
- [P-Flow: Prompting Visual Effects Generation](https://mlsanity.com/paper/2603.22091/) (2026)
- [Semantic Browsing: Controllable Diversity for Image Generation](https://mlsanity.com/paper/2606.23679/) (2026)

## Abstract (arXiv)

The rapid advancement of generative AI has democratized access to powerful tools such as Text-to-Image models. However, to generate high-quality images, users must still craft detailed prompts specifying scene, style, and context-often through multiple rounds of refinement. We propose PromptSculptor, a novel multi-agent framework that automates this iterative prompt optimization process. Our system decomposes the task into four specialized agents that work collaboratively to transform a short, vague user prompt into a comprehensive, refined prompt. By leveraging Chain-of-Thought reasoning, our framework effectively infers hidden context and enriches scene and background details. To iteratively refine the prompt, a self-evaluation agent aligns the modified prompt with the original input, while a feedback-tuning agent incorporates user feedback for further refinement. Experimental results demonstrate that PromptSculptor significantly enhances output quality and reduces the number of iterations needed for user satisfaction. Moreover, its model-agnostic design allows seamless integration with various T2I models, paving the way for industrial applications.