mlsanityData to 7 Oct 2026

Tencent WorkBuddy Bench: A Multi-Domain Coding-Agent Benchmark with Contamination-Resistant Task Construction

arXiv 2607.20911 ↗First posted 23 Jul 2026cs.CL cs.SEcited by 5

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

We introduce Tencent WorkBuddy Bench, a multi-domain evaluation suite for coding agents; this report documents its construction methodology, scoring protocol, and a cross-model leaderboard. At its core is a unified evaluation framework for constructing and running distribution-informed coding-agent tasks across four work domains - Code, Web, Office, and Security. Rather than adapting public issue text, every task is reverse-engineered from a real commit, pull request, or business scenario and rewritten as a short, colloquial, role-played request, so that a task's prompt is not recoverable by web-searching the underlying issue, pull request, or commit thread. Because the dataset is released openly - task directories, environment images, evaluation harness, tests, and reference solutions - contamination resistance rests on this construction together with dataset versioning rather than on secrecy. The four subsets - repository-level engineering, front-end development, office and business workflows, and red-/blue-team security - probe complementary facets of real work, each with its own verification style. All are packaged in a uniform task-directory format and run, under a uniform and reproducible protocol, on two agent harnesses (CodeBuddy Code and Claude Code); the full open release makes the benchmark reproducible end to end and directly auditable, since any third party can re-run each task and inspect its content. Because each subset uses a different scoring instrument, scores are not comparable across subsets and the suite reports no suite-wide average. We report a cross-model leaderboard across several model families.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 15 of its 25 references that are in the map. First posted on arXiv 23 Jul 2026.

Builds on

15 of 25 references.

References · 15

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  4. 2023RepoBench: Benchmarking Repository-Level Code Auto-Completion Systemscited by 346
  5. 2023InterCode: Standardizing and Benchmarking Interactive Coding with Execution Feedbackcited by 229
  6. 2024Cybench: A Framework for Evaluating Cybersecurity Capabilities and Risks of Language Modelscited by 156
  7. 2024NYU CTF Bench: A Scalable Open-Source Benchmark Dataset for Evaluating LLMs in Offensive Securitycited by 90
  8. 2025CVE-Bench: A Benchmark for AI Agents' Ability to Exploit Real-World Web Application Vulnerabilitiescited by 90
  9. 2025OdysseyBench: Evaluating LLM Agents on Long-Horizon Complex Office Application Workflowscited by 54
  10. 2024Interaction2Code: Benchmarking MLLM-based Interactive Webpage Code Generation from Interactive Prototypingcited by 50
  11. 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 31
  12. 2026Workspace-Bench 1.0: Benchmarking AI Agents on Workspace Tasks with Large-Scale File Dependenciescited by 20
3 more
  1. 2025FrontendBench: A Benchmark for Evaluating LLMs on Front-End Development via Automatic Evaluationcited by 17
  2. 2026Vibe Code Bench: Evaluating AI Models on End-to-End Web Application Developmentcited by 7
  3. 2026SpreadsheetBench 2: Evaluating Agents on End-to-End Business Spreadsheet Workflowscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 5.

5 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026EvoCode-Bench: Evaluating Coding Agents in Multi-Turn Iterative Interactions
  2. 2026MOSAIC-Bench: Measuring Compositional Vulnerability Induction in Coding Agents
  3. 2026Inside the Scaffold: A Source-Code Taxonomy of Coding Agent Architectures
  4. 2026REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage
  5. 2026Permission Denied: Policy-Graded Evaluation of Coding Agents in Hardened Environments
  6. 2026Agent Retrieval Bench: Evaluating Repository Context Retrieval for Coding Agents
  7. 2026SWE-Bench Pro Verified: A Reliable Benchmark for Software Engineering Agents
  8. 2026Towards a Risk Assessment of Malicious Skill Files in Coding Agents