mlsanityData to 7 Oct 2026

GLM-5V-Turbo: Toward a Native Foundation Model for Multimodal Agents

arXiv 2604.26752 ↗First posted 29 Apr 2026cs.CVcited by 30

On the map: LLM agents and tool use → Multimodal and embodied agents

Abstract (arXiv)

We present GLM-5V-Turbo, a step toward native foundation models for multimodal agents. As foundation models are increasingly deployed in real environments, agentic capability depends not only on language reasoning, but also on the ability to perceive, interpret, and act over heterogeneous contexts such as images, videos, webpages, documents, GUIs. GLM-5V-Turbo is built around this objective: multimodal perception is integrated as a core component of reasoning, planning, tool use, and execution, rather than as an auxiliary interface to a language model. This report summarizes the main improvements behind GLM-5V-Turbo across model design, multimodal training, reinforcement learning, toolchain expansion, and integration with agent frameworks. These developments lead to strong performance in multimodal coding, visual tool use, and framework-based agentic tasks, while preserving competitive text-only coding capability. More importantly, our development process offers practical insights for building multimodal agents, highlighting the central role of multimodal perception, hierarchical optimization, and reliable end-to-end verification.

Key facts. Cited by 30 papers in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 26 of its 41 references that are in the map. First posted on arXiv 29 Apr 2026.

Builds on

26 of 41 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025MMSearch-Plus: Benchmarking Provenance-Aware Search for Multimodal Browsing Agents29 Aug

Other references · 25

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  3. 2023MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contextscited by 1,582
  4. 2025DINOv3cited by 1,297
  5. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  6. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,127
  7. 2023MVBench: A Comprehensive Multi-modal Video Understanding Benchmarkcited by 1,086
  8. 2023CogAgent: A Visual Language Model for GUI Agentscited by 692
  9. 2026GLM-5: from Vibe Coding to Agentic Engineeringcited by 502
  10. 2024MMMU-Pro: A More Robust Multi-discipline Multimodal Understanding Benchmarkcited by 494
  11. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  12. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 459
13 more
  1. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  2. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  3. 2024Better & Faster Large Language Models via Multi-token Predictioncited by 344
  4. 2020Query-Key Normalization for Transformerscited by 266
  5. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 240
  6. 2024Design2Code: Benchmarking Multimodal Code Generation for Automated Front-End Engineeringcited by 146
  7. 2024MMSearch: Benchmarking the Potential of Large Models as Multi-modal Search Enginescited by 98
  8. 2025SimpleVQA: Multimodal Factuality Evaluation for Multimodal Large Language Modelscited by 61
  9. 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agentscited by 57
  10. 2025PointArena: Probing Multimodal Grounding Through Language-Guided Pointingcited by 30
  11. 2025Advancing vision-language models in front-end development via data synthesiscited by 16
  12. 2026Vision2Web: A Hierarchical Benchmark for Visual Website Development with Agent Verificationcited by 16
  13. 2025UI2Code^N: UI-to-Code Generation as Interactive Visual Optimizationcited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 30.

30 papers.

Sort:

By area:

  1. 2026MMSkills: Towards Multimodal Skills for General Visual Agentscited by 13
  2. 2026MineExplorer: Evaluating Open-World Exploration of MLLM Agents in Minecraftcited by 3
  3. 2026Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoningcited by 2
  4. 2026VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocationcited by 1
  5. 2026SpatialAct: Probing Spatial Reasoning-to-Action Capabilities of VLM Agents in 3D Scenescited by 1
  6. 2026HLL: Can Agents Cross Humanity's Last Line of Verification?cited by 1
  7. 2026Imagine Before You Predict: Interleaved Latent Visual Reasoning for Video Event Predictioncited by 1
  8. 2026Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluationcited by 1
  9. 2026PerceptionRubrics: Calibrating Multimodal Evaluation to Human Perceptioncited by 1
  10. 2026EvoGUI: An Evolution-Aware Benchmark for GUI State-Transition Understandingcited by 1
  11. 2026Invisible Ink Threats: Adversarial Goals Behind Legitimate Tasks in Computer-Use Agentscited by 1
  12. 2026UrbanGround: From Local Perception to Spatial Agency in a Real-Scale Citycited by 1
18 more
  1. 2026UI-Venus-2 Technical Reportcited by 1
  2. 2026FIKA-Bench: From Fine-grained Recognition to Fine-Grained Knowledge Acquisitioncited by 0
  3. 2026Architect-Ant: Editable Automatic Furnishing of Architectural Floor Planscited by 0
  4. 2026Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verificationcited by 0
  5. 2026MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modificationcited by 0
  6. 2026VideoGAIA: A Benchmark for General AI Assistants on Agentic Video Understandingcited by 0
  7. 2026Ancient-Bench: A Comprehensive Multi-millennial, Multi-medium, and Multi-script Benchmark for Ancient Chinese Artifact Text Recognitioncited by 0
  8. 2026Framework and Benchmark for Code-Driven Agentic Testing in Web Developmentcited by 0
  9. 2026Sci-MMR: Benchmarking Multi-Step Evidence-Grounded Scientific Reasoning in Multimodal Agentscited by 0
  10. 2026Inside VLM Chart Reading: Tracing Value Reading from Vertical Bar Charts Across Space and Depthcited by 0
  11. 2026Block-Sparse Attention with Semantic-Geometric Decoupled Routingcited by 0
  12. 2026CCRV-Bench: Constraint-Based Evaluation of Causal Reasoning in Vision-Language Modelscited by 0
  13. 2026Convergence of Practical Muoncited by 0
  14. 2026SciGen-Verifier: A Multimodal Reasoner for Explainable Verification in Scientific Image Generationcited by 0
  15. 2026IVT-Guard: All-in-One Reasoning Model for AI-Generated Content Detectioncited by 0
  16. 2026PhysFieldBench: Can Multimodal Models Understand Physical Fields?cited by 0
  17. 2026Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Modelscited by 0
  18. 2026OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cellscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Survey on Foundations and Frontiers of Multimodal Agentic Frameworks: Techniques and Applications
  2. 2025DeepEyesV2: Toward Agentic Multimodal Model
  3. 2026VisGym: Diverse, Customizable, Scalable Environments for Multimodal Agents
  4. 2025SenseNova-MARS: Empowering Multimodal Agentic Reasoning and Search via Reinforcement Learning
  5. 2025Benchmarking the Generality of Vision-Language-Action Models
  6. 2026Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?
  7. 2025ToolScope: An Agentic Framework for Vision-Guided and Long-Horizon Tool Use
  8. 2025GLM-4.5V and GLM-4.1V-Thinking: Towards Versatile Multimodal Reasoning with Scalable Reinforcement Learning