# T2I-Copilot: A Training-Free Multi-Agent Text-to-Image System for Enhanced Prompt Interpretation and Interactive Generation

arXiv: https://arxiv.org/abs/2507.20536 · HTML: https://mlsanity.com/paper/2507.20536/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 27 of its 36 references that are in the map.
- First posted on arXiv 28 Jul 2025.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 27 of 27)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://mlsanity.com/paper/2303.17580/) (2023)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2301.13826/) (2023)
- [Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Models](https://mlsanity.com/paper/2303.04671/) (2023)
- [Evaluating Text-to-Visual Generation with Image-to-Text Generation](https://mlsanity.com/paper/2404.01291/) (2024)
- [Agent Laboratory: Using LLM Agents as Research Assistants](https://mlsanity.com/paper/2501.04227/) (2025)
- [Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understanding](https://mlsanity.com/paper/2405.08748/) (2024)
- [Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generation](https://mlsanity.com/paper/2402.17245/) (2024)
- [LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Models](https://mlsanity.com/paper/2305.13655/) (2023)
- [Optimizing Prompts for Text-to-Image Generation](https://mlsanity.com/paper/2212.09611/) (2022)
- [Linguistic Binding in Diffusion Models: Enhancing Attribute Correspondence through Attention Map Alignment](https://mlsanity.com/paper/2306.08877/) (2023)
- [A Task is Worth One Word: Learning with Task Prompts for High-Quality Versatile Image Inpainting](https://mlsanity.com/paper/2312.03594/) (2023)
- [GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing](https://mlsanity.com/paper/2407.05600/) (2024)
- [Self-correcting LLM-controlled Diffusion Models](https://mlsanity.com/paper/2311.16090/) (2023)
- [Golden Noise for Diffusion Models: A Learning Framework](https://mlsanity.com/paper/2411.09502/) (2024)
- [Improving Text-to-Image Consistency via Automatic Prompt Optimization](https://mlsanity.com/paper/2403.17804/) (2024)
- [Region-Aware Text-to-Image Generation via Hard Binding and Soft Refinement](https://mlsanity.com/paper/2411.06558/) (2024)
- [Token Merging for Training-Free Semantic Binding in Text-to-Image Synthesis](https://mlsanity.com/paper/2411.07132/) (2024)
- [A Cat Is A Cat (Not A Dog!): Unraveling Information Mix-ups in Text-to-Image Encoders through Causal Analysis and Embedding Optimization](https://mlsanity.com/paper/2410.00321/) (2024)

## Built on by (most cited first, 30 of 33)

- [UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision](https://mlsanity.com/paper/2601.03193/) (2026)
- [Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation](https://mlsanity.com/paper/2602.01756/) (2026)
- [MIRA: Multimodal Iterative Reasoning Agent for Image Editing](https://mlsanity.com/paper/2511.21087/) (2025)
- [Diffusion Probe: Generated Image Result Prediction Using CNN Probes](https://mlsanity.com/paper/2602.23783/) (2026)
- [Show, Don't Tell: Morphing Latent Reasoning into Image Generation](https://mlsanity.com/paper/2602.02227/) (2026)
- [RAISE: Requirement-Adaptive Evolutionary Refinement for Training-Free Text-to-Image Alignment](https://mlsanity.com/paper/2603.00483/) (2026)
- [coDrawAgents: A Multi-Agent Dialogue Framework for Compositional Image Generation](https://mlsanity.com/paper/2603.12829/) (2026)
- [SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation](https://mlsanity.com/paper/2605.08043/) (2026)
- [Guiding What Not to Generate: Automated Negative Prompting for Text-Image Alignment](https://mlsanity.com/paper/2512.07702/) (2025)
- [Embedding-perturbed Exploration Preference Optimization for Flow Models](https://mlsanity.com/paper/2605.15803/) (2026)
- [OctoT2I: A Self-Evolving Agentic Text-to-Image Router](https://mlsanity.com/paper/2606.01803/) (2026)
- [GenRouter: Unified Workflow Routing for Agentic Image Generation](https://mlsanity.com/paper/2608.16721/) (2026)
- [MapReduce LoRA: Advancing the Pareto Front in Multi-Preference Optimization for Generative Models](https://mlsanity.com/paper/2511.20629/) (2025)
- [Generation Navigator: A State-Aware Agentic Framework for Image Generation](https://mlsanity.com/paper/2605.17969/) (2026)
- [MemoGen: Can Past Experience Improve Future Text-to-Image Generation?](https://mlsanity.com/paper/2606.03243/) (2026)
- [EPIC: Efficient Predicate-Guided Inference-Time Control for Compositional Text-to-Image Generation](https://mlsanity.com/paper/2605.11722/) (2026)
- [PerfGuard: A Performance-Aware Agent for Visual Content Generation](https://mlsanity.com/paper/2601.22571/) (2026)
- [SIDiffAgent: Self-Improving Diffusion Agent](https://mlsanity.com/paper/2602.02051/) (2026)
- [ETPDesigner: Multi-Agent Orchestration for Interactive Multimodal Electronic Theater Program](https://mlsanity.com/paper/2607.19947/) (2026)
- [PRISM: Prompt Refinement via Image-grounded Self-rewarding Mechanism for Text-to-Image Generation](https://mlsanity.com/paper/2607.24353/) (2026)
- [Calibrate What You SHIP: Post-Selection Risk Control for Verifier-Guided Text-to-Image Generation](https://mlsanity.com/paper/2608.21748/) (2026)
- [WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing](https://mlsanity.com/paper/2609.05171/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)
- [AGCD: Agent-Guided Cross-Modal Decoding for Weather Forecasting](https://mlsanity.com/paper/2603.15260/) (2026)
- [A Creative Agent is Worth a 64-Token Template](https://mlsanity.com/paper/2603.17895/) (2026)
- [OrchJail: Jailbreaking Tool-Calling Text-to-Image Agents by Orchestration-Guided Fuzzing](https://mlsanity.com/paper/2605.07414/) (2026)
- [Can Text-to-Image Models Draw from the Right Frame of Reference?](https://mlsanity.com/paper/2608.03357/) (2026)
- [Rethinking Camouflage Image Generation towards a Training-Free Paradigm](https://mlsanity.com/paper/2609.14377/) (2026)
- [Don't Read the Log: Execution Traces Contaminate Verifiers in Video-Generation Agents](https://mlsanity.com/paper/2609.28564/) (2026)
- [AdaPilot: Towards Scene-Adaptive Policy Learning for Cross-Generator Text-to-Image Quality Optimization](https://mlsanity.com/paper/2609.29517/) (2026)

## Abstract (arXiv)

Text-to-Image (T2I) generative models have revolutionized content creation but remain highly sensitive to prompt phrasing, often requiring users to repeatedly refine prompts multiple times without clear feedback. While techniques such as automatic prompt engineering, controlled text embeddings, denoising, and multi-turn generation mitigate these issues, they offer limited controllability, or often necessitate additional training, restricting the generalization abilities. Thus, we introduce T2I-Copilot, a training-free multi-agent system that leverages collaboration between (Multimodal) Large Language Models to automate prompt phrasing, model selection, and iterative refinement. This approach significantly simplifies prompt engineering while enhancing generation quality and text-image alignment compared to direct generation. Specifically, T2I-Copilot consists of three agents: (1) Input Interpreter, which parses the input prompt, resolves ambiguities, and generates a standardized report; (2) Generation Engine, which selects the appropriate model from different types of T2I models and organizes visual and textual prompts to initiate generation; and (3) Quality Evaluator, which assesses aesthetic quality and text-image alignment, providing scores and feedback for potential regeneration. T2I-Copilot can operate fully autonomously while also supporting human-in-the-loop intervention for fine-grained control. On GenAI-Bench, using open-source generation models, T2I-Copilot achieves a VQA score comparable to commercial models RecraftV3 and Imagen 3, surpasses FLUX1.1-pro by 6.17% at only 16.59% of its cost, and outperforms FLUX.1-dev and SD 3.5 Large by 9.11% and 6.36%. Code will be released at: https://github.com/SHI-Labs/T2I-Copilot.