WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Image generation and editing models have advanced rapidly, yet remain unreliable when prompts require external world knowledge. Bounded and long-tail parametric knowledge prevents direct or reason-then-generate approaches from recovering the required facts and visual appearances. Existing agentic generation and editing methods mitigate this limitation with retrieval tools, yet remain constrained by insufficient visual verification, overloaded policy models, and weak integration of retrieved textual and visual evidence. To address these limitations, we present WeAgent-MMGenEdit, a full-stack recipe including a multimodal harness, a scalable data construction pipeline, a comprehensive benchmark, and post-training methods for the agent policy and image backend. We first introduce WeAgent-Harness, a multimodal runtime with persistent evidence management and dedicated verification and integration tools that organize retrieved multimodal evidence into a dense carrier. Upon this, we develop a scalable pipeline for prompt synthesis and agentic trajectory collection, yielding 23K supervised trajectories and 14.7K RL tasks with three-layer verifiable checklists. We further introduce WeBench-MMGenEdit, a bilingual benchmark covering both knowledge-intensive image generation and multi-image editing. Finally, a two-sided post-training recipe based on SFT and RL improves the agent policy and image backend. Together, WeAgent-MMGenEdit enables a 30B-total/3B-active policy to outperform similarly sized policy models and approach the performance of a 1T-parameter agent.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 95 of its 100 references that are in the map. First posted on arXiv 4 Sep 2026.
Builds on
95 of 100 references.
Closest ancestors · 15
Same direction, newest first
- 2026GenRouter: Unified Workflow Routing for Agentic Image Generation17 Aug
- 2026Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation6 Jul
- 2026RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation22 Jun
- 2026GenClaw: Code-Driven Agentic Image Generation28 May
- 2026Generation Navigator: A State-Aware Agentic Framework for Image Generation18 May
- 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling30 Apr
- 2026Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis31 Mar
- 2026Gen-Searcher: Reinforcing Agentic Search for Image Generation30 Mar
- 2026Kimi K2.5: Visual Agentic Intelligence2 Feb
- 2026Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation2 Feb
- 2026GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning26 Jan
- 2025T2I-Copilot: A Training-Free Multi-Agent Text-to-Image System for Enhanced Prompt Interpretation and Interactive Generation28 Jul
3 more
Other references · 80
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
- 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
38 more
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
- 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
- 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
- 2025Group Sequence Policy Optimizationcited by 788
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2024OmniGen: Unified Image Generationcited by 388
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
- 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
- 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
- 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
- 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
- 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
- 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
- 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
- 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
- 2025Seedream 3.0 Technical Reportcited by 146
- 2025HunyuanImage 3.0 Technical Reportcited by 138
- 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
- 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131
50 most cited of 80 All 80 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation
- 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation
- 2026WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents
- 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget
- 2025CRAFT: Continuous Reasoning and Agentic Feedback Tuning for Multimodal Text-to-Image Generation
- 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios
- 2026APE: Agentic Prompt Enhancer for Image Generation and Editing
- 2026M3: High-fidelity Text-to-Image Generation via Multi-Modal, Multi-Agent and Multi-Round Visual Reasoning