mlsanityData to 7 Oct 2026

WeAgent-MMGenEdit: A Full-Stack Recipe for Multimodal Agentic Image Generation and Editing

arXiv 2609.05171 ↗First posted 4 Sep 2026cs.CVcited by 0

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

Image generation and editing models have advanced rapidly, yet remain unreliable when prompts require external world knowledge. Bounded and long-tail parametric knowledge prevents direct or reason-then-generate approaches from recovering the required facts and visual appearances. Existing agentic generation and editing methods mitigate this limitation with retrieval tools, yet remain constrained by insufficient visual verification, overloaded policy models, and weak integration of retrieved textual and visual evidence. To address these limitations, we present WeAgent-MMGenEdit, a full-stack recipe including a multimodal harness, a scalable data construction pipeline, a comprehensive benchmark, and post-training methods for the agent policy and image backend. We first introduce WeAgent-Harness, a multimodal runtime with persistent evidence management and dedicated verification and integration tools that organize retrieved multimodal evidence into a dense carrier. Upon this, we develop a scalable pipeline for prompt synthesis and agentic trajectory collection, yielding 23K supervised trajectories and 14.7K RL tasks with three-layer verifiable checklists. We further introduce WeBench-MMGenEdit, a bilingual benchmark covering both knowledge-intensive image generation and multi-image editing. Finally, a two-sided post-training recipe based on SFT and RL improves the agent policy and image backend. Together, WeAgent-MMGenEdit enables a 30B-total/3B-active policy to outperform similarly sized policy models and approach the performance of a 1T-parameter agent.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 95 of its 100 references that are in the map. First posted on arXiv 4 Sep 2026.

Builds on

95 of 100 references.

Closest ancestors · 15

Same direction, newest first

  1. 2026GenRouter: Unified Workflow Routing for Agentic Image Generation17 Aug
  2. 2026Search Beyond What Can Be Taught: Evolving the Knowledge Boundary in Agentic Visual Generation6 Jul
  3. 2026RS-Gen: A Multi-Stage Agentic Framework for Reasoning and Search-Augmented Image Generation22 Jun
  4. 2026GenClaw: Code-Driven Agentic Image Generation28 May
  5. 2026Generation Navigator: A State-Aware Agentic Framework for Image Generation18 May
  6. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modeling30 Apr
  7. 2026Unify-Agent: A Unified Multimodal Agent for World-Grounded Image Synthesis31 Mar
  8. 2026Gen-Searcher: Reinforcing Agentic Search for Image Generation30 Mar
  9. 2026Kimi K2.5: Visual Agentic Intelligence2 Feb
  10. 2026Mind-Brush: Integrating Agentic Cognitive Search and Reasoning into Image Generation2 Feb
  11. 2026GenAgent: Scaling Text-to-Image Generation via Agentic Multimodal Reasoning26 Jan
  12. 2025T2I-Copilot: A Training-Free Multi-Agent Text-to-Image System for Enhanced Prompt Interpretation and Interactive Generation28 Jul
3 more
  1. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use25 May
  2. 2025Magma: A Foundation Model for Multimodal AI Agents18 Feb
  3. 2024GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing8 Jul

Other references · 80

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2022Scalable Diffusion Models with Transformerscited by 5,931
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  9. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  10. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  11. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
38 more
  1. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  2. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,591
  3. 2025Qwen3-VL Technical Reportcited by 2,534
  4. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  6. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,422
  7. 2023HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Facecited by 1,173
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  9. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,010
  10. 2025Qwen-Image Technical Reportcited by 973
  11. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
  12. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  13. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  14. 2025Group Sequence Policy Optimizationcited by 788
  15. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 659
  16. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 615
  17. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  18. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 511
  19. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  20. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
  21. 2024OmniGen: Unified Image Generationcited by 388
  22. 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
  23. 2024OminiControl: Minimal and Universal Control for Diffusion Transformercited by 334
  24. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
  25. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  26. 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
  27. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Modelscited by 260
  28. 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
  29. 2024Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMscited by 224
  30. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 204
  31. 2025In-Context Edit: Enabling Instructional Image Editing with In-Context Generation in Large Scale Diffusion Transformercited by 199
  32. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  33. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
  34. 2025Less-to-More Generalization: Unlocking More Controllability by In-Context Generationcited by 153
  35. 2025Seedream 3.0 Technical Reportcited by 146
  36. 2025HunyuanImage 3.0 Technical Reportcited by 138
  37. 2024Lumina-T2X: Transforming Text into Any Modality, Resolution, and Duration via Flow-based Large Diffusion Transformerscited by 137
  38. 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 131

50 most cited of 80 All 80 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ImAgent: A Unified Multimodal Agent Framework for Test-Time Scalable Image Generation
  2. 2026ToolArtist: Tool-Using Unified Multimodal Models for Agentic Image Generation
  3. 2026WeAgent-MMSearch: Native Text-Vision Interaction for Multimodal Search Agents
  4. 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget
  5. 2025CRAFT: Continuous Reasoning and Agentic Feedback Tuning for Multimodal Text-to-Image Generation
  6. 2026AgentVista: Evaluating Multimodal Agents in Ultra-Challenging Realistic Visual Scenarios
  7. 2026APE: Agentic Prompt Enhancer for Image Generation and Editing
  8. 2026M3: High-fidelity Text-to-Image Generation via Multi-Modal, Multi-Agent and Multi-Round Visual Reasoning