Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.
Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 31 Mar 2026.
Builds on
0 of 0 references.
No references here yet.
Built on by
Citing papers per year: 2026: 16.
16 papers.
Sort:
By area:
- 2026Flow-OPD: On-Policy Distillation for Flow Matching Modelscited by 29
- 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agentscited by 18
- 2026GenClaw: Code-Driven Agentic Image Generationcited by 10
- 2026SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generationcited by 6
- 2026RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generationcited by 5
- 2026Diffusion Model as a Generalist Segmentation Learnercited by 4
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitantscited by 3
- 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generationcited by 3
- 2026Multimodal Large Language Models for Multi-Subject In-Context Image Generationcited by 2
- 2026JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agentscited by 1
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Modelscited by 1
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editingcited by 0
4 more
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratificationcited by 0
- 2026AgenticGen: Reward-Guided Agentic Video Generation for Advertisingcited by 0
- 2026InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographicscited by 0
- 2026InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographics30 Sep 2026
- 2026Stable-MM-R1: Anchoring Multimodal Reasoning Dynamics via Entropy-Guided Stratification7 Sep 2026
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing4 Sep 2026
- 2026AgenticGen: Reward-Guided Agentic Video Generation for Advertising31 Aug 2026
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Models20 Aug 2026
- 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation5 Aug 2026
- 2026JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents26 Jul 2026
- 2026RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation22 Jun 2026
- 2026GenClaw: Code-Driven Agentic Image Generation28 May 2026
- 2026Flow-OPD: On-Policy Distillation for Flow Matching Models8 May 2026
- 2026SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation8 May 2026
4 more
- 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents6 May 2026
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitants30 Apr 2026
- 2026Diffusion Model as a Generalist Segmentation Learner27 Apr 2026
- 2026Multimodal Large Language Models for Multi-Subject In-Context Image Generation8 Apr 2026
- 2026GenClaw: Code-Driven Agentic Image Generationcited by 10
- 2026RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generationcited by 5
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitantscited by 3
- 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generationcited by 3
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editingcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Controlcited by 0
- 2026AgenticGen: Reward-Guided Agentic Video Generation for Advertisingcited by 0
- 2026Agentic Visual Generation: From Generative Models to Agentic Control6 Sep 2026
- 2026WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing4 Sep 2026
- 2026AgenticGen: Reward-Guided Agentic Video Generation for Advertising31 Aug 2026
- 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation5 Aug 2026
- 2026RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation22 Jun 2026
- 2026GenClaw: Code-Driven Agentic Image Generation28 May 2026
- 2026GUI Agents with Reinforcement Learning: Toward Digital Inhabitants30 Apr 2026
- 2026Flow-OPD: On-Policy Distillation for Flow Matching Modelscited by 29
- 2026Diffusion Model as a Generalist Segmentation Learnercited by 4
- 2026Flow-OPD: On-Policy Distillation for Flow Matching Models8 May 2026
- 2026Diffusion Model as a Generalist Segmentation Learner27 Apr 2026
- 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agentscited by 18
- 2026SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generationcited by 6
- 2026JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agentscited by 1
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Modelscited by 1
- 2026InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographicscited by 0
- 2026InfoAgent: Traceable Generation and Repair of Evidence-Grounded Infographics30 Sep 2026
- 2026Exploring the Performance Frontier of Compact Unified Image Generation Models20 Aug 2026
- 2026JarvisHub: An Open Harness for Canvas-Native Multimodal Creative Agents26 Jul 2026
- 2026SCOPE: Structured Decomposition and Conditional Skill Orchestration for Complex Image Generation8 May 2026
- 2026OpenSearch-VL: An Open Recipe for Frontier Multimodal Search Agents6 May 2026
Similar, not cited
Close in wording, no citation either way
- 2026GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning
- 2025A Unified Multi-Agent Framework for Universal Multimodal Understanding and Generation
- 2025ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation
- 2024DiffusionAgent: Navigating Expert Models for Agentic Image Generation
- 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing
- 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget
- 2025OIG-Bench: A Multi-Agent Annotated Benchmark for Multimodal One-Image Guides Understanding
- 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios