mlsanityData to 7 Oct 2026

OSExpert: Computer-Use Agents Learning Professional Skills via Exploration

arXiv 2603.07978 ↗First posted 9 Mar 2026cs.AIcited by 7

On the map: LLM agents and tool use → GUI and mobile agents

Abstract (arXiv)

General-purpose computer-use agents have shown impressive performance across diverse digital environments. However, our new benchmark, OSExpert-Eval, indicates they remain far less helpful than human experts. Although inference-time scaling enables adaptation, these agents complete complex tasks inefficiently with degraded performance, transfer poorly to unseen UIs, and struggle with fine-grained action sequences. To solve the problem, we introduce a GUI-based depth-first search (GUI-DFS) exploration algorithm to comprehensively explore and verify an environment's unit functions. The agent then exploits compositionality between unit skills to self-construct a curriculum for composite tasks. To support fine-grained actions, we curate a database of action primitives for agents to discover during exploration; these are saved as a skill set once the exploration is complete. We use the learned skills to improve the agent's performance and efficiency by (1) enriching agents with ready-to-use procedural knowledge, allowing them to plan only once for long trajectories and generate accurate actions, and (2) enabling them to end inference-time scaling earlier by realizing their boundary of capabilities. Extensive experiments show that our environment-learned agent takes a meaningful step toward expert-level computer use, achieving a around 20 percent performance gain on OSExpert-Eval and closing the efficiency gap to humans by around 80 percent

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use. Builds on 18 of its 29 references that are in the map. First posted on arXiv 9 Mar 2026.

Builds on

18 of 29 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning2 Sep
  2. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Use6 Aug
  3. 2025OSWorld-Human: Benchmarking the Efficiency of Computer-Use Agents19 Jun
  4. 2025macOSWorld: A Multilingual Interactive Benchmark for GUI Agents4 Jun
  5. 2025Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents1 Apr
  6. 2025Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks20 Jan
  7. 2024Agent S: An Open Agentic Framework that Uses Computers Like a Human10 Oct
  8. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale12 Sep
  9. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments11 Apr

Other references · 9

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  2. 2022WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agentscited by 1,197
  3. 2018TextWorld: A Learning Environment for Text-based Gamescited by 304
  4. 2018Reinforcement Learning on Web Interfaces Using Workflow-Guided Explorationcited by 282
  5. 2019Interactive Fiction Games: A Colossal Adventurecited by 165
  6. 2025SEAgent: Self-Evolving Computer Use Agent with Autonomous Learning from Experiencecited by 68
  7. 2024OfficeBench: Benchmarking Language Agents across Multiple Applications for Office Automationcited by 66
  8. 2025WebEvolver: Enhancing Web Agent Self-Improvement with Coevolving World Modelcited by 56
  9. 2025BEARCUBS: A benchmark for computer-using web agentscited by 29

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 7.

7 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026MMG2Skill: Can Agents Distill In-the-Wild Guides into Self-Evolving Skills?
  2. 2025Mirage-1: Augmenting and Updating GUI Agent with Hierarchical Multimodal Skills
  3. 2026ExpVoyager: Direct Experience Navigation for Dynamic Agent Skill Synthesis
  4. 2025GUI Exploration Lab: Enhancing Screen Navigation in Agents via Multi-Turn Reinforcement Learning
  5. 2026OmegaUse: Building a General-Purpose GUI Agent for Autonomous Task Execution
  6. 2025UltraCUA: A Foundation Model for Computer Use Agents with Hybrid Action
  7. 2026Empowering GUI Agents via Autonomous Experience Exploration and Hindsight Experience Utilization for Task Planning
  8. 2026SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation