# Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

arXiv: https://arxiv.org/abs/2601.11868 · HTML: https://mlsanity.com/paper/2601.11868/

## Key facts

- Cited by 585 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jan 2026.

## Built on by (most cited first, 30 of 585)

- [DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligence](https://mlsanity.com/paper/2606.19348/) (2026)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks](https://mlsanity.com/paper/2602.12670/) (2026)
- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [Gemma 4 Technical Report](https://mlsanity.com/paper/2607.02770/) (2026)
- [Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses](https://mlsanity.com/paper/2604.25850/) (2026)
- [Kimi K3: Open Frontier Intelligence](https://mlsanity.com/paper/2607.24653/) (2026)
- [CoEvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification](https://mlsanity.com/paper/2604.01687/) (2026)
- [Self-Harness: Harnesses That Improve Themselves](https://mlsanity.com/paper/2606.09498/) (2026)
- [SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?](https://mlsanity.com/paper/2603.15401/) (2026)
- [Qwen3-Coder-Next Technical Report](https://mlsanity.com/paper/2603.00729/) (2026)
- [How Well Do Agentic Skills Work in the Wild: Benchmarking LLM Skill Usage in Realistic Settings](https://mlsanity.com/paper/2604.04323/) (2026)
- [Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agents](https://mlsanity.com/paper/2604.06132/) (2026)
- [Natural-Language Agent Harnesses](https://mlsanity.com/paper/2603.25723/) (2026)
- [The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence](https://mlsanity.com/paper/2605.26494/) (2026)
- [ProgramBench: Can Language Models Rebuild Programs From Scratch?](https://mlsanity.com/paper/2605.03546/) (2026)
- [Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows](https://mlsanity.com/paper/2605.27922/) (2026)
- [DeepSWE: Measuring Frontier Coding Agents on Original, Long-Horizon Engineering Tasks](https://mlsanity.com/paper/2607.07946/) (2026)
- [Rethinking the Evaluation of Harness Evolution for Agents](https://mlsanity.com/paper/2607.12227/) (2026)
- [TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agents](https://mlsanity.com/paper/2604.24005/) (2026)
- [Code as Agent Harness](https://mlsanity.com/paper/2605.18747/) (2026)
- [On Data Engineering for Scaling LLM Terminal Capabilities](https://mlsanity.com/paper/2602.21193/) (2026)
- [SWE-chat: Coding Agent Interactions From Real Users in the Wild](https://mlsanity.com/paper/2604.20779/) (2026)
- [Nemotron 3 Ultra: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://mlsanity.com/paper/2606.15007/) (2026)
- [OSWorld 2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks](https://mlsanity.com/paper/2606.29537/) (2026)
- [τ-Knowledge: Evaluating Conversational Agents over Unstructured Knowledge](https://mlsanity.com/paper/2603.04370/) (2026)
- [SkillRet: A Large-Scale Benchmark for Skill Retrieval in LLM Agents](https://mlsanity.com/paper/2605.05726/) (2026)
- [Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://mlsanity.com/paper/2604.12374/) (2026)
- [Beyond Semantic Similarity: Rethinking Retrieval for Agentic Search via Direct Corpus Interaction](https://mlsanity.com/paper/2605.05242/) (2026)
- [WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluation](https://mlsanity.com/paper/2605.10912/) (2026)

## Abstract (arXiv)

AI agents may soon become capable of autonomously completing valuable, long-horizon tasks in diverse domains. Current benchmarks either do not measure real-world tasks, or are not sufficiently difficult to meaningfully measure frontier models. To this end, we present Terminal-Bench 2.0: a carefully curated hard benchmark composed of 89 tasks in computer terminal environments inspired by problems from real workflows. Each task features a unique environment, human-written solution, and comprehensive tests for verification. We show that frontier models and agents score less than 65% on the benchmark and conduct an error analysis to identify areas for model and agent improvement. We publish the dataset and evaluation harness to assist developers and researchers in future work at https://www.tbench.ai/ .