mlsanityData to 7 Oct 2026

OS-Symphony: A Holistic Framework for Robust and Generalist Computer-Using Agent

arXiv 2601.07779 ↗First posted 12 Jan 2026cs.MA cs.AI cs.CLcited by 19

On the map: Multimodal LLMs / vision-language models → Embodied and 3D vision-language models

also in 1 more
Abstract (arXiv)

While Vision-Language Models (VLMs) have significantly advanced Computer-Using Agents (CUAs), current frameworks struggle with robustness in long-horizon workflows and generalization in novel domains. These limitations stem from a lack of granular control over historical visual context curation and the absence of visual-aware tutorial retrieval. To bridge these gaps, we introduce OS-Symphony, a holistic framework that comprises an Orchestrator coordinating two key innovations for robust automation: (1) a Reflection-Memory Agent that utilizes milestone-driven long-term memory to enable trajectory-level self-correction, effectively mitigating visual context loss in long-horizon tasks; (2) Versatile Tool Agents featuring a Multimodal Searcher that adopts a SeeAct paradigm to navigate a browser-based sandbox to synthesize live, visually aligned tutorials, thereby resolving fidelity issues in unseen scenarios. Experimental results demonstrate that OS-Symphony delivers substantial performance gains across varying model scales, establishing new state-of-the-art results on three online benchmarks, notably achieving 65.84% on OSWorld.

Key facts. Cited by 19 papers in the map (data through 7 Oct 2026). Placed in Embodied and 3D vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 39 of its 58 references that are in the map. First posted on arXiv 12 Jan 2026.

Builds on

39 of 58 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025MGA: Memory-Driven GUI Agent for Observation-Centric Interaction28 Oct
  2. 2025Retrieval-augmented GUI Agents with Generative Guidelines29 Sep
  3. 2025ScaleCUA: Scaling Open-Source Computer Use Agents with Cross-Platform Data18 Sep
  4. 2024OS-ATLAS: A Foundation Action Model for Generalist GUI Agents30 Oct

Other references · 35

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2025InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiencycited by 1,419
  3. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  4. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  5. 2024SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agentscited by 553
  6. 2024GPT-4V(ision) is a Generalist Web Agent, if Groundedcited by 547
  7. 2024Aguvis: Unified Pure Vision Agents for Autonomous GUI Interactioncited by 271
  8. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 212
  9. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learningcited by 205
  10. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Humancited by 182
  11. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agentscited by 156
  12. 2024OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesiscited by 155
23 more
  1. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 136
  2. 2025Scaling Computer-Use Grounding via User Interface Decomposition and Synthesiscited by 132
  3. 2024AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorialscited by 123
  4. 2025GTA1: GUI Test-time Scaling Agentcited by 102
  5. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  6. 2025DeepAgent: A General Reasoning Agent with Scalable Toolsetscited by 69
  7. 2025ComputerRL: Scaling End-to-End Online Reinforcement Learning for Computer Use Agentscited by 49
  8. 2025PC-Agent: A Hierarchical Multi-Agent Collaboration Framework for Complex Task Automation on PCcited by 39
  9. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  10. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Actioncited by 30
  11. 2025API Agents vs. GUI Agents: Divergence and Convergencecited by 28
  12. 2024AgentStore: Scalable Integration of Heterogeneous Agents As Specialized Generalist Computer Assistantcited by 27
  13. 2025BacktrackAgent: Enhancing GUI Agent with Error Detection and Backtracking Mechanismcited by 25
  14. 2025Game-TARS: Pretrained Foundation Models for Scalable Generalist Multimodal Game Agentscited by 23
  15. 2025WorldGUI: An Interactive Benchmark for Desktop GUI Automation from Any Starting Pointcited by 19
  16. 2025Grounding Computer Use Agents on Human Demonstrationscited by 16
  17. 2025Breaking the Data Barrier -- Building GUI Agents Through Task Generalizationcited by 13
  18. 2025R-WoM: Retrieval-augmented World Model For Computer-use Agentscited by 10
  19. 2025AgentProg: Empowering Long-Horizon GUI Agents with Program-Guided Context Managementcited by 10
  20. 2025Agentic Lybic: Multi-Agent Execution System with Tiered Reasoning and Orchestrationcited by 8
  21. 2025OS-Oracle: A Comprehensive Framework for Cross-Platform GUI Critic Modelscited by 6
  22. 2025Nested Browser-Use Learning for Agentic Information Seekingcited by 5
  23. 2025UI-CUBE: Enterprise-Grade Computer Use Agent Benchmarking Beyond Task Accuracy to Operational Reliabilitycited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 19.

19 papers.

Sort:

By area:

  1. 2026Natural-Language Agent Harnessescited by 54
  2. 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Executioncited by 13
  3. 2026Agent Alpha: Tree Search Unifying Generation, Exploration and Evaluation for Computer-Use Agentscited by 12
  4. 2026OpenMobile: Building Open Mobile Agents with Task and Trajectory Synthesiscited by 10
  5. 2026ToolCUA: Towards Optimal GUI-Tool Path Orchestration for Computer Use Agentscited by 10
  6. 2026OS-Themis: A Scalable Critic Framework for Generalist GUI Rewardscited by 9
  7. 2026GUIDE: Resolving Domain Bias in GUI Agents through Real-Time Web Video Retrieval and Plug-and-Play Annotationcited by 4
  8. 2026VLAA-GUI: Knowing When to Stop, Recover, and Search, A Modular Framework for GUI Automationcited by 4
  9. 2026StructAgent: Harness Long-horizon Digital Agents with Unified Causal Structurecited by 4
  10. 2026MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillationcited by 3
  11. 2026MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktopcited by 2
  12. 2026Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?cited by 1
7 more
  1. 2026A History-Aware Visually Grounded Critic for Computer Use Agentscited by 1
  2. 2026Global Commander and Local Operative: A Dual-Agent Framework for Scene Navigationcited by 0
  3. 2026SynChain: Inducing Computer-Use Agent Systems to Construct Their Own Attack Chainscited by 0
  4. 2026LocalLSTC: A Long Short-Term Control Architecture for Locally Deployed GUI Agentscited by 0
  5. 2026Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimizationcited by 0
  6. 2026AnyAct: Universal Action for Self-Evolving Agentscited by 0
  7. 2026GUI-HARVEST: Self-Improving GUI Agents through Evidence-Driven Harness Evolutioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026OS-SPEAR: A Toolkit for the Safety, Performance,Efficiency, and Robustness Analysis of OS Agents
  2. 2026CUAAudit: Meta-Evaluation of Vision-Language Models as Auditors of Autonomous Computer-Use Agents
  3. 2026MUSE: A Unified Agentic Harness for MLLMs
  4. 2026MintAct: A Unified Visual Agent for Digital Environments
  5. 2026Adaptive Vision-Language Model Routing for Computer Use Agents
  6. 2024MMFactory: A Universal Solution Search Engine for Vision-Language Tasks
  7. 2026RoboHarness: A Memory-Augmented Policy Harness for Vision-Language-Action Model Robustness via In-Context Adaptation
  8. 2024ScreenAgent: A Vision Language Model-driven Computer Control Agent