# WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces

arXiv: https://arxiv.org/abs/2606.09426 · HTML: https://mlsanity.com/paper/2606.09426/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 11 of its 39 references that are in the map.
- First posted on arXiv 8 Jun 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 11 of 11)

- [ReAct: Synergizing Reasoning and Acting in Language Models](https://mlsanity.com/paper/2210.03629/) (2022)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [Terminal-World: Scaling Terminal-Agent Environments via Agent Skills](https://mlsanity.com/paper/2605.20876/) (2026)
- [CocoaBench: Evaluating Unified Digital Agents in the Wild](https://mlsanity.com/paper/2604.11201/) (2026)

## Built on by (most cited first, 9 of 9)

- [LongHorizon-Harness: Advancing Long-Horizon Agents for Real-World Tasks](https://mlsanity.com/paper/2608.01964/) (2026)
- [How Benchmarks Mis-Score Computer-Use Agents](https://mlsanity.com/paper/2607.28367/) (2026)
- [OneDayAgent: Towards a Long-Horizon Harness for Autonomous Agents](https://mlsanity.com/paper/2608.05013/) (2026)
- [UI-Mate: Advancing Open-Weight Foundation GUI Agents with In-Context Demonstrations](https://mlsanity.com/paper/2608.15930/) (2026)
- [RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents](https://mlsanity.com/paper/2609.22000/) (2026)
- [OctoNest: Adaptive Cross-Device Execution through Stateful Control](https://mlsanity.com/paper/2606.20487/) (2026)
- [CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering](https://mlsanity.com/paper/2609.32600/) (2026)
- [Relevance Does Not Imply Applicability: Experience Activation for Personal GUI Agents](https://mlsanity.com/paper/2609.33304/) (2026)
- [HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents](https://mlsanity.com/paper/2609.38008/) (2026)

## Abstract (arXiv)

Computer-use agents (CUAs) increasingly operate in runtimes that combine visual desktop control, command-line execution, code editing, browsers, and external tools. Existing benchmarks, however, often evaluate these interfaces as separable capabilities, leaving long-horizon cross-interface orchestration under-tested. Thus, we introduce WeaveBench, a long-horizon hybrid-interface benchmark with 114 tasks across 8 real-world work domains, grounded in real user requests and publicly verifiable artifacts. Each task requires agents to combine GUI observations/actions with CLI/code operations within a single trajectory. We evaluate these tasks on a real Ubuntu desktop inside deployed CLI-agent runtimes, augmented with a minimal desktop-control plugin. We also propose a companion trajectory-aware judge that inspects deliverables, files, screenshots, logs, and action traces, while detecting shortcut behaviors such as fabricated visual evidence or hard-coded metrics. Across frontier model-runtime pairings, the best PassRate reaches only 41.2%, showing the benchmark remains far from saturated. The trajectory-aware judge further reveals that outcome-only grading substantially overestimates agent performance. Overall, WeaveBench exposes a critical gap in CUA evaluation and provides an effective testbed to measure whether agents can orchestrate GUI, CLI, and code operations across long-horizon real-world tasks.