# Claw-Anything: Benchmarking Always-On Personal Assistants with Broader Access to User's Digital World

arXiv: https://arxiv.org/abs/2605.26086 · HTML: https://mlsanity.com/paper/2605.26086/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in GUI and mobile agents, one of 16 directions in LLM agents and tool use.
- Builds on 13 of its 28 references that are in the map.
- First posted on arXiv 25 May 2026.

## Placed in

- LLM agents and tool use / [GUI and mobile agents](https://mlsanity.com/agents/gui-and-mobile-agents/)

## Builds on (most cited first, 13 of 13)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Training Software Engineering Agents and Verifiers with SWE-Gym](https://mlsanity.com/paper/2412.21139/) (2024)
- [SWE-smith: Scaling Data for Software Engineering Agents](https://mlsanity.com/paper/2504.21798/) (2025)
- [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://mlsanity.com/paper/2604.06132/) (2026)
- [ContextAgent: Context-Aware Proactive LLM Agents with Open-World Sensory Perceptions](https://mlsanity.com/paper/2505.14668/) (2025)
- [FeatureBench: Benchmarking Agentic Coding for Complex Feature Development](https://mlsanity.com/paper/2602.10975/) (2026)
- [ClawBench: Can AI Agents Complete Everyday Online Tasks?](https://mlsanity.com/paper/2604.08523/) (2026)
- [TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agents](https://mlsanity.com/paper/2602.07274/) (2026)
- [Gym-Anything: Turn any Software into an Agent Environment](https://mlsanity.com/paper/2604.06126/) (2026)
- [CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion](https://mlsanity.com/paper/2602.10999/) (2026)
- [ProAgentBench: Evaluating LLM Agents for Proactive Assistance with Real-World Data](https://mlsanity.com/paper/2602.04482/) (2026)
- [SWE Context Bench: A Benchmark for Context Learning in Coding](https://mlsanity.com/paper/2602.08316/) (2026)

## Built on by (most cited first, 1 of 1)

- [PDEU-Bench: Benchmarking the Personalized Planning Lifecycle of Tool-Calling LLM Agents](https://mlsanity.com/paper/2609.34930/) (2026)

## Abstract (arXiv)

Large language model agents are increasingly envisioned as always-on personal assistants with access to anything relevant in the user's digital world. Yet current systems operate over only narrow slices of that world, limiting context-sensitive reasoning and effective assistance. Existing benchmarks similarly provide only partial user state and therefore fail to capture performance in such a broad, always-on setting. To address this gap, we introduce Claw-Anything, a benchmark that expands agent context along three dimensions: long-horizon activity histories, interdependent backend services, and integrated GUI and CLI interaction across multiple devices. To instantiate this setting, we simulate months of user activity through multi-round event injection, producing complex world states and realistic noise, including irrelevant events and conflicting signals. Agents must reason over rich contextual environments while remaining robust to such noise. This expanded scope also enables the evaluation of proactive assistance, requiring agents to anticipate user needs and deliver timely recommendations. Experiments show that GPT-5.5 achieves only 34.5% pass@1, substantially below prior benchmarks, underscoring a gap between current agent capabilities and the demands of always-on personal assistance. Alongside the benchmark, we release an automated data-generation pipeline that yields 2,000 training environments and improves the base model by 23.7%, demonstrating its utility of scalable data infrastructure.