Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Visual generators excel at rendering, but they confidently fabricate what they do not know. User requests are unbounded, evolving, and deeply long-tailed: new characters, trending entities, post-cutoff events, and more. This world-knowledge bottleneck is structural: generators are trained on fixed corpora, but the visual world is open-ended. We construct SearchGen-20K and SearchGen-Bench, with 20,839 prompts spanning twelve failure categories and twenty-two domains, paired with a pre-executed multimodal SearchGen-Corpus-1M to support offline, reproducible research. On SearchGen-Bench, frontier open generators score only 21 to 28 out of 100, a 40-point collapse invisible to existing benchmarks. The natural remedy is to employ search tools, enabling agentic visual generation. However, we find that naive search fails: it retrieves indiscriminately, injecting noise into prompts the generator already handles. We trace the root cause to a generator-specific, evolving knowledge boundary: the divide between what a generator can internalize through training and what must remain in external context. Although this boundary is hard to specify in advance, we show that it is discoverable through a teach-then-search co-training framework. Even a minimal version of this co-training recipe produces monotonic improvement, laying the foundation for recursive self-improvement in visual generation that can meet world-knowledge-grounded requests. We release the full dataset, co-training corpus, and search corpus as a replayable harness for tool-augmented, world-knowledge-grounded visual generation.
Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 42 of its 50 references that are in the map. First posted on arXiv 6 Jul 2026.
Builds on
42 of 50 references.
Closest ancestors · 2
Same direction, newest first
- 2026Gen-Searcher: Reinforcing Agentic Search for Image Generation30 Mar
- 2026GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning26 Jan
Other references · 40
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,839
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
- 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
28 more
- 2022Atlas: Few-shot Learning with Retrieval Augmented Language Modelscited by 952
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 792
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
- 2024Self-Rewarding Language Modelscited by 681
- 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
- 2023Active Retrieval Augmented Generationcited by 516
- 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
- 2024Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexitycited by 451
- 2023BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editingcited by 442
- 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
- 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learningcited by 323
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
- 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 243
- 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
- 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 210
- 2023Holistic Evaluation of Text-To-Image Modelscited by 160
- 2022KNN-Diffusion: Image Generation via Large-Scale Retrievalcited by 137
- 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
- 2024Instruct-Imagen: Image Generation with Multi-modal Instructioncited by 65
- 2024Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratingscited by 44
- 2025Emergent Hierarchical Reasoning in LLMs through Reinforcement Learningcited by 42
- 2024IterComp: Iterative Composition-Aware Feedback Learning from Model Gallery for Text-to-Image Generationcited by 33
- 2025Reverse-Engineered Reasoning for Open-Ended Generationcited by 32
- 2025To Code or not to Code? Adaptive Tool Integration for Math Language Models via Expectation-Maximizationcited by 25
- 2026RationalRewards: Reasoning Rewards Scale Visual Generation Both Training and Test Timecited by 14
- 2026Bad Seeing or Bad Thinking? Rewarding Perception for Multimodal Reasoningcited by 7
Built on by
Citing papers per year: 2026: 8.
8 papers.
- 2026Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillationcited by 4
- 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generationcited by 4
- 2026AVA-Encoder: Towards Agent-Native Video Representation Learningcited by 2
- 2026SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Historiescited by 1
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editingcited by 1
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 1
- 2026The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generationcited by 0
- 2026RSIgen: Reciprocal Self-Improvement for Agentic Visual Generationcited by 0
- 2026RSIgen: Reciprocal Self-Improvement for Agentic Visual Generation13 Oct 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing4 Sep 2026
- 2026The Missing Temporal Link: Temporal Context Routing for Script-Driven Audio-Video Generation2 Sep 2026
- 2026AVA-Encoder: Towards Agent-Native Video Representation Learning12 Aug 2026
- 2026SpecPath: Testing Coding Agents Across Contract-Equivalent Specification Histories10 Aug 2026
- 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation5 Aug 2026
- 2026Rethinking Classifier-Free Guidance in On-Policy Diffusion Distillation27 Jul 2026
Similar, not cited
Close in wording, no citation either way
- 2026MemoGen: Can Past Experience Improve Future Text-to-Image Generation?
- 2026Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation
- 2026GenEvolve: Self-Evolving Image Generation Agents via Tool-Orchestrated Visual Experience Distillation
- 2026OmniHarness: Harnessing Generalizable Visual Generation via Symbolic Policy Learning
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling
- 2025Thinking-while-Generating: Interleaving Textual Reasoning throughout Visual Generation
- 2026Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models
- 2026Diversity Over Frequency: Rethinking Tool Use in Visual Chain-of-Thought Agents