mlsanityData to 14 Oct 2026

Good Benchmarks

arXiv 2607.12217 ↗First posted 13 Jul 2026cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Good tasks are correct, solvable, verifiable, well-specified, and hard for interesting reasons. The best tasks describe a real problem an experienced practitioner would recognize, in language a practitioner would use, with tests that verify the outcome rather than the approach.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 2 of its 7 references that are in the map. First posted on arXiv 13 Jul 2026.

Builds on

2 of 7 references.

References · 2

  1. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  2. 2026Terminal Wrench: A Dataset of 331 Reward-Hackable Environments and 3,632 Exploit Trajectoriescited by 15

Built on by

Not cited yet.