mlsanityData to 7 Oct 2026

ColorBrowserAgent: Complex Long-Horizon Browser Agent with Adaptive Knowledge Evolution

arXiv 2601.07262 ↗First posted 12 Jan 2026cs.HCcited by 6

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

With the advancement of vision-language models, web automation has made significant progress. However, deploying autonomous agents in real-world settings remains challenging, primarily due to site heterogeneity, where generalist models lack domain-specific priors for diverse interfaces, and long-horizon instability, characterized by the accumulation of decision drift over extended interactions. To address these challenges, we introduce ColorBrowserAgent (Complex Long-Horizon Browser Agent), a knowledge-evolving agent for robust web automation. Our approach addresses these challenges through two synergistic mechanisms: human-in-the-loop knowledge adaptation that transforms sparse human feedback into reusable domain knowledge, and knowledge-aligned progressive summarization that stabilizes long interactions through memory compression. Extensive experiments on WebArena, WebChoreArena and industrial deployment show that ColorBrowserAgent consistently outperforms strong baselines. It achieves a state-of-the-art success rate of 71.2% on WebArena and maintains 47.4% performance under zero-shot transfer setting on WebChoreArena. In commercial deployment, it improves user satisfaction by 19.3% relatively, verifying its robustness in real-world scenarios.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 20 of its 31 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

20 of 31 references.

References · 20

  1. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  2. 2023AppAgent: Multimodal Agents as Smartphone Userscited by 407
  3. 2024WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learningcited by 243
  4. 2021A Survey of Human-in-the-loop for Machine Learningcited by 214
  5. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  6. 2025An Illusion of Progress? Assessing the Current State of Web Agentscited by 165
  7. 2025WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learningcited by 162
  8. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  9. 2024The BrowserGym Ecosystem for Web Agent Researchcited by 152
  10. 2024AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agentscited by 110
  11. 2024WebPilot: A Versatile and Autonomous Multi-Agent System for Web Task Execution with Strategic Explorationcited by 100
  12. 2025A Survey of WebAgents: Towards Next-Generation AI Agents for Web Automation with Large Foundation Modelscited by 87
8 more
  1. 2025WebExplorer: Explore and Evolve for Training Long-Horizon Web Agentscited by 86
  2. 2024AppAgent v2: Advanced Agent for Flexible Mobile Interactionscited by 67
  3. 2023SteP: Stacked LLM Policies for Web Actionscited by 66
  4. 2017Human-in-the-loop Artificial Intelligencecited by 40
  5. 2025Go-Browse: Training Web Agents with Structured Explorationcited by 32
  6. 2024BPP-Search: Enhancing Tree of Thought Reasoning for Mathematical Modeling Problem Solvingcited by 15
  7. 2025WebOperator: Action-Aware Tree Search for Autonomous Agents in Web Environmentcited by 10
  8. 2025WebDART: Dynamic Decomposition and Re-planning for Complex Web Taskscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 6.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025BrowserAgent: Building Web Agents with Human-Inspired Web Browsing Actions
  2. 2024AdaptAgent: Adapting Multimodal Web Agents with Few-Shot Learning from Human Demonstrations
  3. 2023A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis
  4. 2026CAP: A Scalable Benchmark for Evaluating Cross-Site Browser Agents with Complex Actions and Perception
  5. 2025WebSight: A Vision-First Architecture for Robust Web Agents
  6. 2026MolmoWeb: Open Visual Web Agent and Open Data for the Open Web
  7. 2024VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks
  8. 2026Structured Interaction, Visual Localization, and Robust Execution for Complex Web Tasks: A Technical Report on the WebRetriever Challenge