mlsanityData to 7 Oct 2026

Surfer 2: The Next Generation of Cross-Platform Computer Use Agents

arXiv 2510.19949 ↗First posted 22 Oct 2025cs.AIcited by 5

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

Building agents that generalize across web, desktop, and mobile environments remains an open challenge, as prior systems rely on environment-specific interfaces that limit cross-platform deployment. We introduce Surfer 2, a unified architecture operating purely from visual observations that achieves state-of-the-art performance across all three environments. Surfer 2 integrates hierarchical context management, decoupled planning and execution, and self-verification with adaptive recovery, enabling reliable operation over long task horizons. Our system achieves 97.1% accuracy on WebVoyager, 69.6% on WebArena, 60.1% on OSWorld, and 87.1% on AndroidWorld, outperforming all prior systems without task-specific fine-tuning. With multiple attempts, Surfer 2 exceeds human performance on all benchmarks. These results demonstrate that systematic orchestration amplifies foundation model capabilities and enables general-purpose computer control through visual interaction alone, while calling for a next-generation vision language model to achieve Pareto-optimal cost-efficiency.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 47 of its 57 references that are in the map. First posted on arXiv 22 Oct 2025.

Builds on

47 of 57 references.

Closest ancestors · 17

Same direction, newest first

  1. 2025Scaling Agents for Computer Use2 Oct
  2. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  3. 2025Mobile-Agent-v3: Fundamental Agents for GUI Automation21 Aug
  4. 2025OpenCUA: Open Foundations for Computer-Use Agents12 Aug
  5. 2025MagicGUI: A Foundational Mobile GUI Agent with Scalable Data Pipeline and Reinforcement Fine-tuning19 Jul
  6. 2025Efficient Agent Training for Computer Use20 May
  7. 2025Digi-Q: Learning Q-Value Functions for Training Device-Control Agents13 Feb
  8. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agents21 Jan
  9. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  10. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Human10 Oct
  11. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale12 Sep
  12. 2024DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning14 Jun
5 more
  1. 2024On the Effects of Data Scale on UI Control Agents6 Jun
  2. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr
  3. 2024Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception29 Jan
  4. 2023CogAgent: A Visual Language Model for GUI Agents14 Dec
  5. 2023You Only Look at Screens: Multimodal Chain-of-Action Agents20 Sep

Other references · 30

  1. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,655
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  5. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  6. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  7. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  8. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 450
  9. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Modelscited by 424
  10. 2023Android in the Wild: A Large-Scale Dataset for Android Device Controlcited by 314
  11. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  12. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
18 more
  1. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  2. 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
  3. 2024Agent Q: Advanced Reasoning and Learning for Autonomous AI Agentscited by 201
  4. 2025Plan-and-Act: Improving Planning of Agents for Long-Horizon Taskscited by 190
  5. 2022Offline RL for Natural Language Generation with Implicit Language Q Learningcited by 137
  6. 2024Autonomous Evaluation and Refinement of Digital Agentscited by 136
  7. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 119
  8. 2024Vision Language Models are In-Context Value Learnerscited by 84
  9. 2024Agent-E: From Autonomous Web Navigation to Foundational Design Principles in Agentic Systemscited by 79
  10. 2022ScreenQA: Large-Scale Question-Answer Pairs over Mobile App Screenshotscited by 65
  11. 2024Offline Reinforcement Learning for LLM Multi-Step Reasoningcited by 53
  12. 2024Proposer-Agent-Evaluator(PAE): Autonomous Skill Discovery For Foundation Model Internet Agentscited by 50
  13. 2022CHAI: A CHatbot AI for Task-Oriented Dialogue with Offline Reinforcement Learningcited by 33
  14. 2024Enhancing Decision-Making for LLM Agents via Step-Level Q-Value Modelscited by 28
  15. 2024Beyond Sparse Rewards: Enhancing Reinforcement Learning with Language Model Critique in Text Generationcited by 22
  16. 2024ReAct Meets ActRe: When Language Agents Enjoy Training Data Autonomycited by 22
  17. 2025Mano Technical Reportcited by 13
  18. 2025Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weightscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026MintAct: A Unified Visual Agent for Digital Environments
  2. 2025MobileUse: A GUI Agent with Hierarchical Reflection for Autonomous Mobile Operation
  3. 2025WebSight: A Vision-First Architecture for Robust Web Agents
  4. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents
  5. 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
  6. 2026DevicesWorld: Benchmarking Cross-Device Agents in Heterogeneous Environments
  7. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action
  8. 2026OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent