Terminal Agents: A Survey of AI Agents in Command-Line Environments
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Large language model agents increasingly act through terminals, yet existing surveys disperse terminal-mediated behavior across software engineering, tool use, and computer-use research. We regard terminal agents as systems whose dominant progress-bearing action--observation loop is mediated by terminal command execution, textual feedback, and stateful environment interaction. Using terminal-mediated execution as an organizing lens, this survey establishes workload-level boundaries and connects system architecture, competence acquisition, and evaluation through a seven-dimensional terminal competence profile. Our synthesis shows that realized behavior is jointly shaped by the model, interface, harness, runtime, and environment. Executable trajectories ground learning in action consequences, verification, and recovery, whereas prevailing evaluations emphasize final outcomes and expose process quality, recovery, and governance unevenly. Bounded fixed-condition diagnostics illustrate two implications: benchmark families expose different process signals, and matched system comparisons reveal benchmark-dependent performance and limits of component attribution. These findings motivate explicit reporting of system and runtime conditions, supported by replayable traces and process-level evidence. The framework provides a unified basis for studying terminal-mediated agency across software engineering and emerging application domains.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 135 of its 191 references that are in the map. First posted on arXiv 20 Aug 2026.
Builds on
135 of 191 references.
Closest ancestors · 57
Same direction, newest first
- 2026The Scaffolding Matters More Than the Interface: A Controlled Comparison of MCP and CLI Tool Use Across Seven Agent Scaffoldings, Five Language Models, and One Software Task9 Aug
- 2026DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds6 Aug
- 2026Permission Denied: Policy-Graded Evaluation of Coding Agents in Hardened Environments2 Aug
- 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectories10 Jul
- 2026Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions2 Jul
- 2026The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation8 Jun
- 2026Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops8 Jun
- 2026ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agents18 May
- 2026Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety12 May
- 2026AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems9 May
- 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
- 2026AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking26 Apr
38 more
- 2026NetAgentBench: A State-Centric Benchmark for Evaluating Agentic Network Configuration3 Apr
- 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure2 Apr
- 2026REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage2 Apr
- 2026Agent psychometrics: Task-level performance prediction in agentic coding benchmarks1 Apr
- 2026Terminal Agents Suffice for Enterprise Automation31 Mar
- 2026SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks25 Mar
- 2026Effective Strategies for Asynchronous Software Engineering Agents23 Mar
- 2026AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling22 Mar
- 2026SWE-Next: Scalable Real-World Software Engineering Tasks for Agents21 Mar
- 2026When the Specification Emerges: Benchmarking Faithfulness Loss in Long-Horizon Coding Agents17 Mar
- 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?16 Mar
- 2026Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned5 Mar
- 2026SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks28 Feb
- 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale27 Feb
- 2026RepoMod-Bench: A Benchmark for Code Repository Modernization via Implementation-Agnostic Testing26 Feb
- 2026ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads?23 Feb
- 2026CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion11 Feb
- 2026AgentStepper: Interactive Debugging of Software Development Agents6 Feb
- 2026SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training3 Feb
- 2026ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development2 Feb
- 2026Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering1 Feb
- 2026Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments1 Feb
- 2026DockSmith: Scaling Reliable Coding Environments via an Agentic Docker Builder31 Jan
- 2026daVinci-Dev: Agent-native Mid-training for Software Engineering26 Jan
- 2026OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding15 Jan
- 2026Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub1 Jan
- 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios20 Dec
- 2025The Dual-State Architecture for Reliable LLM Agents18 Dec
- 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents14 Dec
- 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?17 Nov
- 2025The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents5 Nov
- 2025Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation10 Oct
- 2025Where LLM Agents Fail and How They can Learn From Failures29 Sep
- 2025Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents27 Sep
- 2025Towards Agentic OS: An LLM Agent Framework for Linux Schedulers1 Sep
- 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities15 Aug
- 2025A Survey on Code Generation with LLM-based Agents31 Jul
- 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments11 Jul
50 newest of 57 All 57 (JSON)
Other references · 78
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
- 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
- 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
- 2025Survey on Evaluation of LLM-based Agentscited by 204
- 2022TALM: Tool Augmented Language Modelscited by 197
- 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 148
- 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 127
- 2024AIOS: LLM Agent Operating Systemcited by 97
- 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 78
- 2026AutoHarness: improving LLM agents by automatically synthesizing a code harnesscited by 76
38 more
- 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 74
- 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agentscited by 64
- 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 54
- 2025ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Taskscited by 51
- 2026On Data Engineering for Scaling LLM Terminal Capabilitiescited by 43
- 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 37
- 2026WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluationcited by 37
- 2025Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learningcited by 36
- 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
- 2025Towards Trustworthy GUI Agents: A Surveycited by 34
- 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
- 2025Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agentscited by 33
- 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 33
- 2026Tmax: A simple recipe for terminal agentscited by 33
- 2026Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Gradingcited by 31
- 2026TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agentscited by 29
- 2025DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecyclecited by 23
- 2026ECHO: Terminal Agents Learn World Models for Freecited by 19
- 2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 19
- 2026Quantifying Frontier LLM Capabilities for Container Sandbox Escapecited by 18
- 2026AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agentscited by 18
- 2025ML-Dev-Bench: Comparative Analysis of AI Agents on ML development workflowscited by 17
- 2025BashArena: A Control Setting for Highly Privileged AI Agentscited by 17
- 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agentscited by 17
- 2026ClawSafety: "Safe" LLMs, Unsafe Agentscited by 17
- 2025SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Developmentcited by 16
- 2025Git Context Controller: Manage the Context of LLM-based Agents like Gitcited by 15
- 2026A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compressioncited by 15
- 2026ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Contextcited by 14
- 2025IRCopilot: Automated Incident Response with Large Language Modelscited by 13
- 2025From Reproduction to Replication: Evaluating Research Agents with Progressive Code Maskingcited by 12
- 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agentscited by 12
- 2026CLI-Universe: Towards Verifiable Task Synthesis Engine for Terminal Agentscited by 12
- 2026SETA: Scaling Environments for Terminal Agentscited by 12
- 2025Secure and Efficient Access Control for Computer-Use Agents via Context Spacecited by 11
- 2026TerminalWorld: Benchmarking Agents on Real-World Terminal Taskscited by 10
- 2025SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Taskscited by 9
- 2026LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarioscited by 9
50 most cited of 78 All 78 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skills
- 2026Scaling Agentic Data for Long-Horizon Terminal Intelligence
- 2026FACET: Preserving Source Intent and Executable State in Terminal Task Synthesis
- 2026AgentAbstain: Do LLM Agents Know When Not to Act?
- 2026Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress
- 2026LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces
- 2026Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification
- 2026Toward Scalable Terminal Task Synthesis via Skill Graphs