mlsanityData to 14 Oct 2026

A Survey on Agentic Multimodal Large Language Models

arXiv 2510.10991 ↗First posted 13 Oct 2025cs.CV cs.AI cs.CLcited by 23

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

With the recent emergence of revolutionary autonomous agentic systems, research community is witnessing a significant shift from traditional static, passive, and domain-specific AI agents toward more dynamic, proactive, and generalizable agentic AI. Motivated by the growing interest in agentic AI and its potential trajectory toward AGI, we present a comprehensive survey on Agentic Multimodal Large Language Models (Agentic MLLMs). In this survey, we explore the emerging paradigm of agentic MLLMs, delineating their conceptual foundations and distinguishing characteristics from conventional MLLM-based agents. We establish a conceptual framework that organizes agentic MLLMs along three fundamental dimensions: (i) Agentic internal intelligence functions as the system's commander, enabling accurate long-horizon planning through reasoning, reflection, and memory; (ii) Agentic external tool invocation, whereby models proactively use various external tools to extend their problem-solving capabilities beyond their intrinsic knowledge; and (iii) Agentic environment interaction further situates models within virtual or physical environments, allowing them to take actions, adapt strategies, and sustain goal-directed behavior in dynamic real-world scenarios. To further accelerate research in this area for the community, we compile open-source training frameworks, training and evaluation datasets for developing agentic MLLMs. Finally, we review the downstream applications of agentic MLLMs and outline future research directions for this rapidly evolving field. To continuously track developments in this rapidly evolving field, we will also actively update a public repository at https://github.com/HJYao00/Awesome-Agentic-MLLMs.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 343 of its 396 references that are in the map. First posted on arXiv 13 Oct 2025.

Builds on

343 of 396 references.

Closest ancestors · 38

Same direction, newest first

  1. 2025Towards Efficient Multimodal Unified Reasoning Model via Model Merging10 Oct
  2. 2025Look Again, Think Slowly: Enhancing Visual Reflection in Vision-Language Models15 Sep
  3. 2025Igniting VLMs toward the Embodied Space15 Sep
  4. 2025OmniEVA: Embodied Versatile Planner via Task-Adaptive 3D-Grounded and Embodiment-aware Reasoning11 Sep
  5. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning1 Jul
  6. 2025GRPO-CARE: Consistency-Aware Reinforcement Learning for Multimodal Reasoning19 Jun
  7. 2025Reinforcing Spatial Reasoning in Vision-Language Models with Interwoven Thinking and Visual Drawing11 Jun
  8. 2025GUI-Reflection: Empowering Multimodal GUI Models with Self-Reflection Behavior9 Jun
  9. 2025Look Before You Leap: A GUI-Critic-R1 Model for Pre-Operative Error Diagnosis in GUI Automation5 Jun
  10. 2025Advancing Multimodal Reasoning: From Optimized Cold Start to Staged Reinforcement Learning4 Jun
  11. 2025MiMo-VL Technical Report4 Jun
  12. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning2 Jun
26 more
  1. 2025GThinker: Towards General Multimodal Reasoning via Cue-Guided Rethinking1 Jun
  2. 2025ZeroGUI: Automating Online GUI Learning at Zero Human Cost29 May
  3. 2025SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward22 May
  4. 2025R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO22 May
  5. 2025Adaptive Chain-of-Focus Reasoning via Dynamic Visual Search and Zooming for Efficient VLMs21 May
  6. 2025DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning20 May
  7. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
  8. 2025EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learning7 May
  9. 2025Fast-Slow Thinking GRPO for Large Vision-Language Model Reasoning25 Apr
  10. 2025InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners19 Apr
  11. 2025NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation17 Apr
  12. 2025GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents14 Apr
  13. 2025SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models10 Apr
  14. 2025VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning10 Apr
  15. 2025VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model10 Apr
  16. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  17. 2025Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning23 Mar
  18. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization17 Mar
  19. 2025LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL10 Mar
  20. 2025Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning10 Mar
  21. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models9 Mar
  22. 2024AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations20 Nov
  23. 2024Improve Vision Language Model Chain-of-thought Reasoning21 Oct
  24. 2024GUI-World: A Video Benchmark and Dataset for Multimodal GUI-oriented Understanding16 Jun
  25. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models25 Jan
  26. 2023ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation24 Dec

Other references · 305

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  5. 2025Qwen3 Technical Reportcited by 8,820
  6. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2023Visual Instruction Tuningcited by 7,869
  9. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  10. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  11. 2025Qwen2.5-VL Technical Reportcited by 5,587
  12. 2024GPT-4o System Cardcited by 4,588
38 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  3. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  7. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  8. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  9. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  10. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  11. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  12. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  13. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
  14. 2025InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Modelscited by 1,834
  15. 2025Gemma 3 Technical Reportcited by 1,824
  16. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  17. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  18. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,614
  19. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,510
  20. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  21. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  22. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  23. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,444
  24. 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,402
  25. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  26. 2023LLaVA-Med: Training a Large Language-and-Vision Assistant for Biomedicine in One Daycited by 1,305
  27. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,251
  28. 2023Mind2Web: Towards a Generalist Agent for the Webcited by 1,242
  29. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
  30. 2024DeepSeek-Coder: When the Large Language Model Meets Programming -- The Rise of Code Intelligencecited by 1,181
  31. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  32. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,079
  33. 2025A-MEM: Agentic Memory for LLM Agentscited by 1,063
  34. 2024DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Modelscited by 867
  35. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  36. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 838
  37. 2024Evaluating Very Long-Term Conversational Memory of LLM Agentscited by 830
  38. 2025Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memorycited by 779

50 most cited of 305 All 305 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 20.

23 papers.

Sort:

By area:

  1. 2025MedSAM3: Delving into Segment Anything with Medical Conceptscited by 26
  2. 2026MM-DeepResearch: A Simple and Effective Multimodal Agentic Search Baselinecited by 20
  3. 2026Toward Efficient Agents: Memory, Tool learning, and Planningcited by 16
  4. 2025CaptionQA: Is Your Caption as Useful as the Image Itself?cited by 15
  5. 2026The Vision Wormhole: Latent-Space Communication in Heterogeneous Multi-Agent Systemscited by 9
  6. 2025VG-Refiner: Towards Tool-Refined Referring Grounded Reasoning via Agentic Reinforcement Learningcited by 8
  7. 2026VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoningcited by 8
  8. 2026DeepImageSearch: Benchmarking Multimodal Agents for Context-Aware Image Retrieval in Visual Historiescited by 6
  9. 2026ProactiveMobile: A Comprehensive Benchmark for Boosting Proactive Intelligence on Mobile Devicescited by 6
  10. 2026From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AIcited by 6
  11. 2026Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agentscited by 5
  12. 2026Scaling Video Understanding via Compact Latent Multi-Agent Collaborationcited by 4
11 more
  1. 2026EvoDiagram: Agentic Editable Diagram Creation via Design Expertise Evolutioncited by 2
  2. 2026M3-ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineeringcited by 1
  3. 2026HippoCamp: Benchmarking Contextual Agents on Personal Computerscited by 1
  4. 2026Can We Defend Against AI-Generated Video Attacks on Real-World Crisis Events? A Systematic Evaluation of Detectors, Generators and Social Disseminationcited by 1
  5. 2026UrbanGround: From Local Perception to Spatial Agency in a Real-Scale Citycited by 1
  6. 2026R1-SyntheticVL: Is Synthetic Data from Generative Models Ready for Multimodal Large Language Model?cited by 0
  7. 2026Large Language Models in Transportation Systems Management and Operations: From Text Reasoning to Multi-modal Decision Supportcited by 0
  8. 2026Understanding and Exploiting Diagonal Attention Sparsity in Autoregressive Image Generationcited by 0
  9. 2026One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMscited by 0
  10. 2026MLLMs Fail to Refuse when Using Tools Agenticallycited by 0
  11. 2026PLAYGROUND: Progressive Layout Generation with Render-Grounded Planningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026The Path Ahead for Agentic AI: Challenges and Opportunities
  2. 2026Agentic Artificial Intelligence (AI): Architectures, Taxonomies, and Evaluation of Large Language Model Agents
  3. 2026Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?
  4. 2025Towards Embodied Agentic AI: Review and Classification of LLM- and VLM-Driven Robot Autonomy and Interaction
  5. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agents
  6. 2025From Language to Action: A Review of Large Language Models as Autonomous Agents and Tool Users
  7. 2023The Rise and Potential of Large Language Model Based Agents: A Survey
  8. 2024Large Language Model based Multi-Agents: A Survey of Progress and Challenges