mlsanityData to 7 Oct 2026

Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis

arXiv 2603.29620 ↗First posted 31 Mar 2026cs.CV cs.MMcited by 16

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Unified multimodal models provide a natural and promising architecture for understanding diverse and complex real-world knowledge while generating high-quality images. However, they still rely primarily on frozen parametric knowledge, which makes them struggle with real-world image generation involving long-tail and knowledge-intensive concepts. Inspired by the broad success of agents on real-world tasks, we explore agentic modeling to address this limitation. Specifically, we present Unify-Agent, a unified multimodal agent for world-grounded image synthesis, which reframes image generation as an agentic pipeline consisting of prompt understanding, multimodal evidence searching, grounded recaptioning, and final synthesis. To train our model, we construct a tailored multimodal data pipeline and curate 143K high-quality agent trajectories for world-grounded image synthesis, enabling effective supervision over the full agentic generation process. We further introduce FactIP, a benchmark covering 12 categories of culturally significant and long-tail factual concepts that explicitly requires external knowledge grounding. Extensive experiments show that our proposed Unify-Agent substantially improves over its base unified model across diverse benchmarks and real world generation tasks, while approaching the world knowledge capabilities of the strongest closed-source models. As an early exploration of agent-based modeling for world-grounded image synthesis, our work highlights the value of tightly coupling reasoning, searching, and generation for reliable open-world agentic image synthesis.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. First posted on arXiv 31 Mar 2026.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 16.

16 papers.

Sort:

By area:

Similar, not cited

Close in wording, no citation either way

  1. 2026GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning
  2. 2025A Unified Multi-Agent Framework for Universal Multimodal Understanding and Generation
  3. 2025ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation
  4. 2024DiffusionAgent: Navigating Expert Models for Agentic Image Generation
  5. 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing
  6. 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget
  7. 2025OIG-Bench: A Multi-Agent Annotated Benchmark for Multimodal One-Image Guides Understanding
  8. 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios