mlsanityData to 14 Oct 2026

Surfer-H Meets Holo1: Cost-Efficient Web Agent Powered by Open Weights

arXiv 2506.02865 ↗First posted 3 Jun 2025cs.AIcited by 8

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

We present Surfer-H, a cost-efficient web agent that integrates Vision-Language Models (VLM) to perform user-defined tasks on the web. We pair it with Holo1, a new open-weight collection of VLMs specialized in web navigation and information extraction. Holo1 was trained on carefully curated data sources, including open-access web content, synthetic examples, and self-produced agentic data. Holo1 tops generalist User Interface (UI) benchmarks as well as our new web UI localization benchmark, WebClick. When powered by Holo1, Surfer-H achieves a 92.2% state-of-the-art performance on WebVoyager, striking a Pareto-optimal balance between accuracy and cost-efficiency. To accelerate research advancement in agentic systems, we are open-sourcing both our WebClick evaluation dataset and the Holo1 model weights.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 17 of its 33 references that are in the map. First posted on arXiv 3 Jun 2025.

Builds on

17 of 33 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025ScreenSpot-Pro: GUI Grounding for Professional High-Resolution Computer Use4 Apr
  2. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct
  3. 2024Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents7 Oct

Other references · 14

  1. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  3. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  4. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  5. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,745
  6. 2018Object Hallucination in Image Captioningcited by 669
  7. 2025UI-TARS: Pioneering Automated GUI Interaction with Native Agentscited by 610
  8. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 567
  9. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  10. 2022ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detectioncited by 422
  11. 2024What matters when building vision-language models?cited by 326
  12. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
2 more
  1. 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 75
  2. 2024Can Language Models Solve Olympiad Programming?cited by 72

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 4.

8 papers.

  1. 2025From Prompt Injections to Protocol Exploits: Threats in LLM-Powered AI Agents Workflowscited by 58
  2. 2025How to Train Your LLM Web Agent: A Statistical Diagnosiscited by 19
  3. 2026OpenWebRL: Demystifying Online Multi-turn Reinforcement Learning for Visual Web Agentscited by 6
  4. 2025Surfer 2: The Next Generation of Cross-Platform Computer Use Agentscited by 5
  5. 2026Weblica: Scalable and Reproducible Training Environments for Visual Web Agentscited by 4
  6. 2025A Multimodal GUI Architecture for Interfacing with LLM-Based Conversational Assistantscited by 1
  7. 2026Best-of-Q: Improving VLM agents with Q-function Action Ranking at Inferencecited by 0
  8. 2026CAPTCHAs in the Agentic Era: Solvers That Learn from Every Encountercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
  2. 2024WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models
  3. 2023Multimodal Web Navigation with Instruction-Finetuned Foundation Models
  4. 2025WebSight: A Vision-First Architecture for Robust Web Agents
  5. 2026OpAgent: Operator Agent for Web Navigation
  6. 2024OpenWebVoyager: Building Multimodal Web Agents via Iterative Real-World Exploration, Feedback and Optimization
  7. 2026WebChallenger: A Reliable and Efficient Generalist Web Agent
  8. 2023Mind2Web: Towards a Generalist Agent for the Web