Surfer 2: The Next Generation of Cross-Platform Computer Use Agents
On the map: LLM agents and tool use → GUI and mobile agents
Abstract (arXiv)
Building agents that generalize across web, desktop, and mobile environments remains an open challenge, as prior systems rely on environment-specific interfaces that limit cross-platform deployment. We introduce Surfer 2, a unified architecture operating purely from visual observations that achieves state-of-the-art performance across all three environments. Surfer 2 integrates hierarchical context management, decoupled planning and execution, and self-verification with adaptive recovery, enabling reliable operation over long task horizons. Our system achieves 97.1% accuracy on WebVoyager, 69.6% on WebArena, 60.1% on OSWorld, and 87.1% on AndroidWorld, outperforming all prior systems without task-specific fine-tuning. With multiple attempts, Surfer 2 exceeds human performance on all benchmarks. These results demonstrate that systematic orchestration amplifies foundation model capabilities and enables general-purpose computer control through visual interaction alone, while calling for a next-generation vision language model to achieve Pareto-optimal cost-efficiency.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 47 of its 57 references that are in the map. First posted on arXiv 22 Oct 2025.
Builds on
47 of 57 references.
Closest ancestors · 17
Same direction, newest first
- 2025Scaling Agents for Computer Use2 Oct
- 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
- 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation21 Aug
- 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
- 2025MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning19 Jul
- 2025Efficient Agent Training for Computer Use20 May
- 2025Digi-Q: Learning Q-Value Functions for Training Device-Control Agents13 Feb
- 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
- 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
- 2024Agent S: An Open Agentic Framework that Uses Computers Like a Human10 Oct
- 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale12 Sep
- 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning14 Jun
5 more
- 2024On the Effects of Data Scale on UI Control Agents6 Jun
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
- 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception29 Jan
- 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
- 2023You Only Look at Screens: Multimodal Chain-of-Action Agents20 Sep
Other references · 30
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
- 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
- 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
- 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
- 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
- 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
- 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
- 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
18 more
- 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
- 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
- 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
- 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
- 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 137
- 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
- 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
- 2024Vision Language Models are In-Context Value Learnerscited by 84
- 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 79
- 2022ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshotscited by 65
- 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 53
- 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
- 2022CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learningcited by 33
- 2024Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Modelscited by 28
- 2024Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generationcited by 22
- 2024ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomycited by 22
- 2025Mano Technical Reportcited by 13
- 2025Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weightscited by 8
Built on by
Citing papers per year: 2026: 5.
5 papers.
- 2026Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agentscited by 5
- 2026Synergy: A Next-Generation General-Purpose Agent for Open Agentic Webcited by 2
- 2026Agent-Computer Observation Interfaces Enable Dynamic Computer Usecited by 1
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliabilitycited by 0
- 2026DragOn: A Benchmark and Dataset for Drag-Based GUI Interactionscited by 0
- 2026Learning from Failure: Inference-Time Self-Improvement for Computer-Use Agents30 Jun 2026
- 2026Agent-Computer Observation Interfaces Enable Dynamic Computer Use28 Jun 2026
- 2026DragOn: A Benchmark and Dataset for Drag-Based GUI Interactions4 Jun 2026
- 2026Securing Computer-Use Agents: A Unified Architecture-Lifecycle Framework for Deployment-Grounded Reliability8 May 2026
- 2026Synergy: A Next-Generation General-Purpose Agent for Open Agentic Web30 Mar 2026
Similar, not cited
Close in wording, no citation either way
- 2026MintAct: A Unified Visual Agent for Digital Environments
- 2025MobileUse: A GUI Agent with Hierarchical Reflection for Autonomous Mobile Operation
- 2025WebSight: A Vision-First Architecture for Robust Web Agents
- 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
- 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
- 2026DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments
- 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action
- 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent