mlsanityData to 7 Oct 2026

Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

arXiv 2601.11868 ↗First posted 17 Jan 2026cs.SE cs.AIcited by 515

Not on the map. Why?

Cited by 341 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not sufficiently difficult to meaningfully measure frontier models. To this end, we present Terminal-Bench 2.0: a carefully curated hard benchmark composed of 89 tasks in computer terminal environments inspired by problems from real workflows. Each task features a unique environment, human-written solution, and comprehensive tests for verification. We show that frontier models and agents score less than 65% on the benchmark and conduct an error analysis to identify areas for model and agent improvement. We publish the dataset and evaluation harness to assist developers and researchers in future work at https://www.tbench.ai/ .

Key facts. Cited by 515 papers in the map (data through 7 Oct 2026). First posted on arXiv 17 Jan 2026.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 515.

515 papers.

Sort:

By area:

  1. 2026Kimi K2.5: Visual Agentic Intelligencecited by 449
  2. 2026SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Taskscited by 289
  3. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
  4. 2026Gemma 4 Technical Reportcited by 187
  5. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnessescited by 119
  6. 2026CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verificationcited by 117
  7. 2026Kimi K3: Open Frontier Intelligencecited by 99
  8. 2026Self-Harness: Harnesses That Improve Themselvescited by 79
  9. 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?cited by 72
  10. 2026How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settingscited by 62
  11. 2026Qwen3-Coder-Next Technical Reportcited by 60
  12. 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agentscited by 57
38 more
  1. 2026Natural-Language Agent Harnessescited by 54
  2. 2026The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligencecited by 50
  3. 2026ProgramBench: Can Language Models Rebuild Programs From Scratch?cited by 47
  4. 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflowscited by 44
  5. 2026DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Taskscited by 41
  6. 2026Code as Agent Harnesscited by 40
  7. 2026Rethinking the Evaluation of Harness Evolution for Agentscited by 39
  8. 2026TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agentscited by 38
  9. 2026SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agentscited by 37
  10. 2026SWE-chat: Coding Agent Interactions From Real Users in the Wildcited by 36
  11. 2026τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledgecited by 35
  12. 2026On Data Engineering for Scaling LLM Terminal Capabilitiescited by 34
  13. 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 33
  14. 2026Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interactioncited by 33
  15. 2026Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 32
  16. 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 31
  17. 2026LongCat-Next: Lexicalizing Modalities as Discrete Tokenscited by 30
  18. 2026WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluationcited by 29
  19. 2026OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Taskscited by 29
  20. 2026Harness Updating Is Not Harness Benefit: Disentangling Evolution Capabilities in Self-Evolving LLM Agentscited by 28
  21. 2026Stop Comparing LLM Agents Without Disclosing the Harnesscited by 25
  22. 2026Agents' Last Examcited by 25
  23. 2026SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Taskscited by 24
  24. 2026ContextBench: A Benchmark for Context Retrieval in Coding Agentscited by 23
  25. 2026From Static Templates to Dynamic Runtime Graphs: A Survey of Workflow Optimization for LLM Agentscited by 22
  26. 2026Qwen-AgentWorld: Language World Models for General Agentscited by 22
  27. 2026From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flawscited by 21
  28. 2026CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversioncited by 20
  29. 2026Composer 2 Technical Reportcited by 20
  30. 2026Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Gradingcited by 20
  31. 2026SkillFlow:Benchmarking Lifelong Skill Discovery and Evolution for Autonomous Agentscited by 19
  32. 2026SWE-Marathon: Can Agents Autonomously Complete Ultra-Long-Horizon Software Work?cited by 19
  33. 2026Cordon: Semantic Transactions for Tool-Using LLM Agentscited by 19
  34. 2026SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Trainingcited by 18
  35. 2026SWE-CI: Evaluating Agent Capabilities in Maintaining Codebases via Continuous Integrationcited by 18
  36. 2026SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolutioncited by 18
  37. 2026RHO: Your Coding Agent is Secretly a Roboticistcited by 18
  38. 2026LLM Agents Already Know When to Call Tools -- Even Without Reasoningcited by 17

50 most cited of 515 All 515 (JSON)