mlsanityData to 14 Oct 2026

CUBE: A Standard for Unifying Agent Benchmarks

arXiv 2603.15798 ↗First posted 16 Mar 2026cs.AIcited by 2

Not on the map. Why?

Abstract (arXiv)

The proliferation of agent benchmarks has created critical fragmentation that threatens research productivity. Each new benchmark requires substantial custom integration, creating an "integration tax" that limits comprehensive evaluation. We propose CUBE (Common Unified Benchmark Environments), a universal protocol standard built on MCP and Gym that allows benchmarks to be wrapped once and used everywhere. By separating task, benchmark, package, and registry concerns into distinct API layers, CUBE enables any compliant platform to access any compliant benchmark for evaluation, RL training, or data generation without custom integration. We call on the community to contribute to the development of this standard before platform-specific implementations deepen fragmentation as benchmark production accelerates through 2026.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 24 of its 40 references that are in the map. First posted on arXiv 16 Mar 2026.

Builds on

24 of 40 references.

References · 24

  1. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  2. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
  3. 2023AgentBench: Evaluating LLMs as Agentscited by 1,267
  4. 2023GAIA: a benchmark for General AI Assistantscited by 1,128
  5. 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
  6. 2024AndroidWorld: A Dynamic Benchmarking Environment for Autonomous Agentscited by 468
  7. 2024AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agentscited by 401
  8. 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 363
  9. 2024AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agentscited by 339
  10. 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
  11. 2024Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scalecited by 223
  12. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 152
12 more
  1. 2025MCP-Bench: Benchmarking Tool-Using LLM Agents with Complex Real-World Tasks via MCP Serverscited by 109
  2. 2025AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectoriescited by 108
  3. 2025The Art of Scaling Reinforcement Learning Compute for LLMscited by 101
  4. 2025MCP-Universe: Benchmarking Large Language Models with Real-World Model Context Protocol Serverscited by 96
  5. 2025MAPoRL: Multi-Agent Post-Co-Training for Collaborative Large Language Models with Reinforcement Learningcited by 86
  6. 2024SPA-Bench: A Comprehensive Benchmark for SmartPhone Agent Evaluationcited by 81
  7. 2024GTA: A Benchmark for General Tool Agentscited by 77
  8. 2024LLM With Tools: A Surveycited by 61
  9. 2024Prompting Large Language Models to Tackle the Full Software Development Lifecycle: A Case Studycited by 44
  10. 2026Gaia2: Benchmarking LLM Agents on Dynamic and Asynchronous Environmentscited by 33
  11. 2025MCPVerse: An Expansive, Real-World Benchmark for Agentic Tool Usecited by 16
  12. 2026General Agent Evaluationcited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026AI-NativeBench: An Open-Source White-Box Agentic Benchmark Suite for AI-Native Systems
  2. 2026MacAgentBench: Benchmarking AI Agents on Real-World macOS Desktop
  3. 2026MCPEvol-Bench: Benchmarking LLM Agent Performance Across Dynamic Evolutions of MCP Servers
  4. 2026DuMateBench: Evaluating Autonomous Agents in Complex Real-World Workflows
  5. 2026AgentSLABench: Evaluating and Benchmarking Agentic Systems Under Resource Constraints
  6. 2026PERFOPT-Bench: Evaluating Coding Agents on Software Performance Optimization
  7. 2026LiveClawBench: Benchmarking LLM Agents on Complex, Real-World Assistant Tasks
  8. 2026Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AI