# ClawMark: A Living-World Benchmark for Multi-Turn, Multi-Day, Multimodal Coworker Agents

arXiv: https://arxiv.org/abs/2604.23781 · HTML: https://mlsanity.com/paper/2604.23781/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use.
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 26 Apr 2026.

## Placed in

- LLM agents and tool use / [General LLM agent frameworks and benchmarks](https://mlsanity.com/agents/general-llm-agent-frameworks-and-benchmarks/)

## Builds on (most cited first, 18 of 18)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://mlsanity.com/paper/2307.13854/) (2023)
- [AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation](https://mlsanity.com/paper/2308.08155/) (2023)
- [MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework](https://mlsanity.com/paper/2308.00352/) (2023)
- [SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering](https://mlsanity.com/paper/2405.15793/) (2024)
- [τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains](https://mlsanity.com/paper/2406.12045/) (2024)
- [CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Society](https://mlsanity.com/paper/2303.17760/) (2023)
- [AgentBench: Evaluating LLMs as Agents](https://mlsanity.com/paper/2308.03688/) (2023)
- [Mind2Web: Towards a Generalist Agent for the Web](https://mlsanity.com/paper/2306.06070/) (2023)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://mlsanity.com/paper/2404.07972/) (2024)
- [Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces](https://mlsanity.com/paper/2601.11868/) (2026)
- [WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?](https://mlsanity.com/paper/2403.07718/) (2024)
- [TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks](https://mlsanity.com/paper/2412.14161/) (2024)
- [MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Servers](https://mlsanity.com/paper/2508.20453/) (2025)
- [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://mlsanity.com/paper/2604.06132/) (2026)
- [MCPMark: A Benchmark for Stress-Testing Realistic and Comprehensive MCP Use](https://mlsanity.com/paper/2509.24002/) (2025)
- [ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspaces](https://mlsanity.com/paper/2604.05172/) (2026)
- [ClawArena: Benchmarking AI Agents in Evolving Information Environments](https://mlsanity.com/paper/2604.04202/) (2026)

## Built on by (most cited first, 17 of 17)

- [Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows](https://mlsanity.com/paper/2605.27922/) (2026)
- [Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependencies](https://mlsanity.com/paper/2605.03596/) (2026)
- [ClawGym: A Scalable Framework for Building Effective Claw Agents](https://mlsanity.com/paper/2604.26904/) (2026)
- [Auditing Agent Harness Safety](https://mlsanity.com/paper/2605.14271/) (2026)
- [Claw-SWE-Bench: A Benchmark for Evaluating OpenClaw-style Agent Harnesses on Coding Tasks](https://mlsanity.com/paper/2606.12344/) (2026)
- [From Chatbot to Digital Colleague: The Paradigm Shift Toward Persistent Autonomous AI](https://mlsanity.com/paper/2606.14502/) (2026)
- [VibeLifeBench: Can Your Life Agent Be Proactive and Persistent in a Living World?](https://mlsanity.com/paper/2608.10875/) (2026)
- [UniClawBench: A Universal Benchmark for Proactive Agents on Real-World Tasks](https://mlsanity.com/paper/2607.08768/) (2026)
- [VibeSearchBench: Benchmarking Long-horizon Proactive Search in the Wild](https://mlsanity.com/paper/2605.27882/) (2026)
- [PolyWorkBench: Benchmarking LLM Agents for Cross-Lingual Long-Horizon Workflows](https://mlsanity.com/paper/2607.06008/) (2026)
- [OmniaBench: Benchmarking General AI Agents Across Diverse Scenarios](https://mlsanity.com/paper/2607.14989/) (2026)
- [Unified Hallucination Fuzzing for Multimodal Large Language Models](https://mlsanity.com/paper/2608.07525/) (2026)
- [DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows](https://mlsanity.com/paper/2608.26546/) (2026)
- [Unified Agent: Managing Interactions across Devices](https://mlsanity.com/paper/2608.05729/) (2026)
- [ClawProBench: Trace-Aware Evaluation of AI Agents with Runtime Coverage and Frozen Workplace-Style Holdouts](https://mlsanity.com/paper/2608.22510/) (2026)
- [WorkWorlds: An Infrastructure for Evaluating AI Agents on Workplace Tasks](https://mlsanity.com/paper/2609.23806/) (2026)
- [Cowork-Gym: Synthesizing Interactive Work Environments at Scale](https://mlsanity.com/paper/2610.17670/) (2026)

## Abstract (arXiv)

Language-model agents are increasingly used as persistent coworkers that assist users across multiple working days. During such workflows, the surrounding environment may change independently of the agent: new emails arrive, calendar entries shift, knowledge-base records are updated, and evidence appears across images, scanned PDFs, audio, video, and spreadsheets. Existing benchmarks do not adequately evaluate this setting because they typically run within a single static episode and remain largely text-centric. We introduce bench, a benchmark for coworker agents built around multi-turn multi-day tasks, a stateful sandboxed service environment whose state evolves between turns, and rule-based verification. The current release contains 100 tasks across 13 professional scenarios, executed against five stateful sandboxed services (filesystem, email, calendar, knowledge base, spreadsheet) and scored by 1537 deterministic Python checkers over post-execution service state; no LLM-as-judge is invoked during scoring. We benchmark seven frontier agent systems. The strongest model reaches 75.8 weighted score, but the best strict Task Success is only 20.0%, indicating that partial progress is common while complete end-to-end workflow completion remains rare. Turn-level analysis shows that performance drops after the first exogenous environment update, highlighting adaptation to changing state as a key open challenge. We release the benchmark, evaluation harness, and construction pipeline to support reproducible coworker-agent evaluation.