# World-To-Image: Grounding Text-to-Image Generation with Agent-Driven World Knowledge

arXiv: https://arxiv.org/abs/2510.04201 · HTML: https://mlsanity.com/paper/2510.04201/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 25 references that are in the map.
- First posted on arXiv 5 Oct 2025.

## Builds on (most cited first, 23 of 23)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022)
- [OmniGen2: Towards Instruction-Aligned Multimodal Generation](https://mlsanity.com/paper/2506.18871/) (2025)
- [Optimizing Prompts for Text-to-Image Generation](https://mlsanity.com/paper/2212.09611/) (2022)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [Playground v3: Improving Text-to-Image Alignment with Deep-Fusion Large Language Models](https://mlsanity.com/paper/2409.10695/) (2024)
- [Improving Text-to-Image Consistency via Automatic Prompt Optimization](https://mlsanity.com/paper/2403.17804/) (2024)
- [PromptMagician: Interactive Prompt Engineering for Text-to-Image Creation](https://mlsanity.com/paper/2307.09036/) (2023)
- [Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?](https://mlsanity.com/paper/2406.07546/) (2024)
- [Prompt Stealing Attacks Against Text-to-Image Generation Models](https://mlsanity.com/paper/2302.09923/) (2023)
- [Dynamic Prompt Optimizing for Text-to-Image Generation](https://mlsanity.com/paper/2404.04095/) (2024)
- [On the Scalability of Diffusion-based Text-to-Image Generation](https://mlsanity.com/paper/2404.02883/) (2024)
- [PALP: Prompt Aligned Personalization of Text-to-Image Models](https://mlsanity.com/paper/2401.06105/) (2024)
- [WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation](https://mlsanity.com/paper/2505.01490/) (2025)
- [Minority-Focused Text-to-Image Generation via Prompt Optimization](https://mlsanity.com/paper/2410.07838/) (2024)
- [Test-time Prompt Refinement for Text-to-Image Models](https://mlsanity.com/paper/2507.22076/) (2025)
- [DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation](https://mlsanity.com/paper/2312.14216/) (2023)
- [CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systems](https://mlsanity.com/paper/2506.08071/) (2025)
- [Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2506.16853/) (2025)

## Built on by (most cited first, 5 of 5)

- [Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation](https://mlsanity.com/paper/2602.01756/) (2026)
- [Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis](https://mlsanity.com/paper/2603.29620/) (2026)
- [SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation](https://mlsanity.com/paper/2605.08043/) (2026)
- [WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing](https://mlsanity.com/paper/2609.05171/) (2026)
- [Agentic Visual Generation: From Generative Models to Agentic Control](https://mlsanity.com/paper/2609.06758/) (2026)

## Abstract (arXiv)

While text-to-image (T2I) models can synthesize high-quality images, their performance degrades significantly when prompted with novel or out-of-distribution (OOD) entities due to inherent knowledge cutoffs. We introduce World-To-Image, a novel framework that bridges this gap by empowering T2I generation with agent-driven world knowledge. We design an agent that dynamically searches the web to retrieve images for concepts unknown to the base model. This information is then used to perform multimodal prompt optimization, steering powerful generative backbones toward an accurate synthesis. Critically, our evaluation goes beyond traditional metrics, utilizing modern assessments like LLMGrader and ImageReward to measure true semantic fidelity. Our experiments show that World-To-Image substantially outperforms state-of-the-art methods in both semantic alignment and visual aesthetics, achieving +8.1% improvement in accuracy-to-prompt on our curated NICE benchmark. Our framework achieves these results with high efficiency in less than three iterations, paving the way for T2I systems that can better reflect the ever-changing real world. Our demo code is available herefootnotehttps://github.com/mhson-kyle/World-To-Image.