MintAct: A Unified Visual Agent for Digital Environments
On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models
also in 1 more
Abstract (arXiv)
We present MintAct, a family of vision-language models that unifies UI grounding, multi-step navigation across mobile, desktop, and web, and visual tool use, trained at 2B, 4B, and 8B scales. Through careful design of our environments, data, and training recipes, MintAct models match the performance of per-domain specialists across all of these capabilities. To enable this, we develop a scalable environment and reinforcement learning (RL) infrastructure. On the environment side, we host hundreds of concurrent instances across heterogeneous per-domain backends, serving both trajectory data collection and online RL. To enable efficient and scalable RL training, an asynchronous framework keeps explicit control over the cross-domain training distribution and remains stable under noisy environment feedback and off-policy drift. Experimental results show that MintAct achieves state-of-the-art performance (48.9 on OSWorld-Verified) across a wide range of benchmarks at comparable model sizes.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 59 references that are in the map. First posted on arXiv 18 Sep 2026.
Builds on
47 of 59 references.
Closest ancestors · 6
Same direction, newest first
- 2025Efficient Multi-turn RL for GUI Agents via Decoupled Training and Adaptive Data Curation28 Sep
- 2025ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data18 Sep
- 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
- 2025Scalable Video-to-Dataset Generation for Cross-Platform Mobile Agents19 May
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning13 May
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
Other references · 41
- 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
- 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,868
- 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,195
- 2023SGLang: Efficient Execution of Structured Language Model Programscited by 1,141
- 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
- 2023AppAgent: Multimodal Agents as Smartphone Userscited by 415
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 319
29 more
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perceptioncited by 293
- 2025AReaL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoningcited by 250
- 2023GPT4Tools: Teaching Large Language Model to Use Tools via Self-instructioncited by 248
- 2024On the Effects of Data Scale on UI Control Agentscited by 194
- 2024GUIOdyssey: A Comprehensive Dataset for Cross-App GUI Navigation on Mobile Devicescited by 194
- 2024Mobile-Agent-v2: Mobile Device Operation Assistant with Effective Navigation via Multi-Agent Collaborationcited by 189
- 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 174
- 2025Thyme: Think Beyond Imagescited by 167
- 2024Aria-UI: Visual Grounding for GUI Instructionscited by 140
- 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 139
- 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
- 2025MAI-UI Technical Report: Real-World Centric Foundation GUI Agentscited by 91
- 2025MobileWorld: Benchmarking Autonomous Mobile Agents in Agent-User Interactive and MCP-Augmented Environmentscited by 72
- 2025MMBench-GUI: Hierarchical Multi-Platform Evaluation Framework for GUI Agentscited by 67
- 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Frameworkcited by 62
- 2026EvoCUA: Evolving Computer Use Agents via Learning from Scalable Synthetic Experiencecited by 53
- 2025SkyRL-Agent: Efficient RL Training for Multi-turn LLM Agentcited by 48
- 2025AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Trainingcited by 41
- 2025InSTA: Towards Internet-Scale Training For Agentscited by 32
- 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Actioncited by 31
- 2026Agents' Last Examcited by 31
- 2025OSWorld-MCP: Benchmarking MCP Tool Invocation In Computer-Use Agentscited by 29
- 2025Grounding Computer Use Agents on Human Demonstrationscited by 22
- 2026MolmoWeb: Open Visual Web Agent and Open Data for the Open Webcited by 19
- 2025Ferret-UI Lite: Lessons from Building Small On-Device GUI Agentscited by 16
- 2026MolmoPoint: Better Pointing for VLMs with Grounding Tokenscited by 15
- 2026iOSWorld: A Benchmark for Personally Intelligent Phone Agentscited by 7
- 2026Weblica: Scalable and Reproducible Training Environments for Visual Web Agentscited by 4
- 2026MM-ToolSandBox: A Unified Framework for Evaluating Visual Tool-Calling Agentscited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Surfer 2: The Next Generation of Cross-Platform Computer Use Agents
- 2025VEM: Environment-Free Exploration for Training GUI Agent with Value Environment Model
- 2024Mobility VLA: Multimodal Instruction Navigation with Long-Context VLMs and Topological Graphs
- 2026Scaling, Benchmarking, and Reasoning of Vision-Language Agents for Mobile GUI Navigation
- 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent
- 2023Scaling Data Generation in Vision-and-Language Navigation
- 2026See and Remember: A Multimodal Agent for Web Traversal
- 2025Mobile-R1: Towards Interactive Capability for VLM-Based Mobile Agent via Systematic Training