mlsanityData to 14 Oct 2026

Terminal Agents: A Survey of AI Agents in Command-Line Environments

arXiv 2608.20485 ↗First posted 20 Aug 2026cs.AI cs.SEcited by 1

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Large language model agents increasingly act through terminals, yet existing surveys disperse terminal-mediated behavior across software engineering, tool use, and computer-use research. We regard terminal agents as systems whose dominant progress-bearing action--observation loop is mediated by terminal command execution, textual feedback, and stateful environment interaction. Using terminal-mediated execution as an organizing lens, this survey establishes workload-level boundaries and connects system architecture, competence acquisition, and evaluation through a seven-dimensional terminal competence profile. Our synthesis shows that realized behavior is jointly shaped by the model, interface, harness, runtime, and environment. Executable trajectories ground learning in action consequences, verification, and recovery, whereas prevailing evaluations emphasize final outcomes and expose process quality, recovery, and governance unevenly. Bounded fixed-condition diagnostics illustrate two implications: benchmark families expose different process signals, and matched system comparisons reveal benchmark-dependent performance and limits of component attribution. These findings motivate explicit reporting of system and runtime conditions, supported by replayable traces and process-level evidence. The framework provides a unified basis for studying terminal-mediated agency across software engineering and emerging application domains.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 135 of its 191 references that are in the map. First posted on arXiv 20 Aug 2026.

Builds on

135 of 191 references.

Closest ancestors · 57

Same direction, newest first

  1. 2026The Scaffolding Matters More Than the Interface: A Controlled Comparison of MCP and CLI Tool Use Across Seven Agent Scaffoldings, Five Language Models, and One Software Task9 Aug
  2. 2026DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds6 Aug
  3. 2026Permission Denied: Policy-Graded Evaluation of Coding Agents in Hardened Environments2 Aug
  4. 2026Failure as a Process: An Anatomy of CLI Coding Agent Trajectories10 Jul
  5. 2026Coding Agents Are Guessing: Measuring Action-Boundary Violations in Underspecified DevOps Instructions2 Jul
  6. 2026The Scaffold Effect in Coding Agents: Harness Choice as a Hidden Variable in Coding-Agent Evaluation8 Jun
  7. 2026Hardening Agent Benchmarks with Adversarial Hacker-Fixer Loops8 Jun
  8. 2026ProcCtrlBench: Evaluating Process-Level Defects and Control Preservation in LLM Coding Agents18 May
  9. 2026Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safety12 May
  10. 2026AgentForesight: Online Auditing for Early Failure Prediction in Multi-Agent Systems9 May
  11. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
  12. 2026AgentEval: DAG-Structured Step-Level Evaluation for Agentic Workflows with Error Propagation Tracking26 Apr
38 more
  1. 2026NetAgentBench: A State-Centric Benchmark for Evaluating Agentic Network Configuration3 Apr
  2. 2026Beyond Resolution Rates: Behavioral Drivers of Coding Agent Success and Failure2 Apr
  3. 2026REAP: Automatic Curation of Coding Agent Benchmarks from Interactive Production Usage2 Apr
  4. 2026Agent psychometrics: Task-level performance prediction in agentic coding benchmarks1 Apr
  5. 2026Terminal Agents Suffice for Enterprise Automation31 Mar
  6. 2026SlopCodeBench: Benchmarking How Coding Agents Degrade Over Long-Horizon Iterative Tasks25 Mar
  7. 2026Effective Strategies for Asynchronous Software Engineering Agents23 Mar
  8. 2026AgentHER: Hindsight Experience Replay for LLM Agent Trajectory Relabeling22 Mar
  9. 2026SWE-Next: Scalable Real-World Software Engineering Tasks for Agents21 Mar
  10. 2026When the Specification Emerges: Benchmarking Faithfulness Loss in Long-Horizon Coding Agents17 Mar
  11. 2026SWE-Skills-Bench: Do Agent Skills Actually Help in Real-World Software Engineering?16 Mar
  12. 2026Building Effective AI Coding Agents for the Terminal: Scaffolding, Harness, Context Engineering, and Lessons Learned5 Mar
  13. 2026SWE-Hub: A Unified Production System for Scalable, Executable Software Engineering Tasks28 Feb
  14. 2026SWE-rebench V2: Language-Agnostic SWE Task Collection at Scale27 Feb
  15. 2026RepoMod-Bench: A Benchmark for Code Repository Modernization via Implementation-Agnostic Testing26 Feb
  16. 2026ISO-Bench: Can Coding Agents Optimize Real-World Inference Workloads?23 Feb
  17. 2026CLI-Gym: Scalable CLI Task Generation via Agentic Environment Inversion11 Feb
  18. 2026AgentStepper: Interactive Debugging of Software Development Agents6 Feb
  19. 2026SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training3 Feb
  20. 2026ProjDevBench: Benchmarking AI Coding Agents on End-to-End Project Development2 Feb
  21. 2026Agyn: A Multi-Agent System for Team-Based Autonomous Software Engineering1 Feb
  22. 2026Large-Scale Terminal Agentic Trajectory Generation from Dockerized Environments1 Feb
  23. 2026DockSmith: Scaling Reliable Coding Environments via an Agentic Docker Builder31 Jan
  24. 2026daVinci-Dev: Agent-native Mid-training for Software Engineering26 Jan
  25. 2026OctoBench: Benchmarking Scaffold-Aware Instruction Following in Repository-Grounded Agentic Coding15 Jan
  26. 2026Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub1 Jan
  27. 2025SWE-EVO: Benchmarking Coding Agents in Long-Horizon Software Evolution Scenarios20 Dec
  28. 2025The Dual-State Architecture for Reliable LLM Agents18 Dec
  29. 2025NL2Repo-Bench: Towards Long-Horizon Repository Generation Evaluation of Coding Agents14 Dec
  30. 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?17 Nov
  31. 2025The OpenHands Software Agent SDK: A Composable and Extensible Foundation for Production Agents5 Nov
  32. 2025Saving SWE-Bench: A Benchmark Mutation Approach for Realistic Agent Evaluation10 Oct
  33. 2025Where LLM Agents Fail and How They can Learn From Failures29 Sep
  34. 2025Kimi-Dev: Agentless Training as Skill Prior for SWE-Agents27 Sep
  35. 2025Towards Agentic OS: An LLM Agent Framework for Linux Schedulers1 Sep
  36. 2025AI Agentic Programming: A Survey of Techniques, Challenges, and Opportunities15 Aug
  37. 2025A Survey on Code Generation with LLM-based Agents31 Jul
  38. 2025SetupBench: Assessing Software Engineering Agents' Ability to Bootstrap Development Environments11 Jul

50 newest of 57 All 57 (JSON)

Other references · 78

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 1,040
  3. 2026Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfacescited by 585
  4. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
  5. 2025SWE-Bench Pro: Can AI Agents Solve Long-Horizon Software Engineering Tasks?cited by 279
  6. 2025Survey on Evaluation of LLM-based Agentscited by 204
  7. 2022TALM: Tool Augmented Language Modelscited by 197
  8. 2025Scaling Long-Horizon LLM Agent via Context-Foldingcited by 148
  9. 2025Memento: Fine-tuning LLM Agents without Fine-tuning LLMscited by 127
  10. 2024AIOS: LLM Agent Operating Systemcited by 97
  11. 2025LifelongAgentBench: Evaluating LLM Agents as Lifelong Learnerscited by 78
  12. 2026AutoHarness: improving LLM agents by automatically synthesizing a code harnesscited by 76
38 more
  1. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 74
  2. 2026Claw-Eval: Towards Trustworthy Evaluation of Autonomous Agentscited by 64
  3. 2026Endless Terminals: Scaling RL Environments for Terminal Agentscited by 54
  4. 2025ITBench: Evaluating AI Agents across Diverse Real-World IT Automation Taskscited by 51
  5. 2026On Data Engineering for Scaling LLM Terminal Capabilitiescited by 43
  6. 2025EXP-Bench: Can AI Conduct AI Research Experiments?cited by 37
  7. 2026WildClawBench: A Benchmark for Real-World, Long-Horizon Agent Evaluationcited by 37
  8. 2025Training Long-Context, Multi-Turn Software Engineering Agents with Reinforcement Learningcited by 36
  9. 2025OS Agents: A Survey on MLLM-based Agents for General Computing Devices Usecited by 35
  10. 2025Towards Trustworthy GUI Agents: A Surveycited by 34
  11. 2025A Survey on Efficient Vision-Language-Action Modelscited by 34
  12. 2025Curie: Toward Rigorous and Automated Scientific Experimentation with AI Agentscited by 33
  13. 2026ClawsBench: Evaluating Capability and Safety of LLM Productivity Agents in Simulated Workspacescited by 33
  14. 2026Tmax: A simple recipe for terminal agentscited by 33
  15. 2026Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Gradingcited by 31
  16. 2026TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agentscited by 29
  17. 2025DAComp: Benchmarking Data Agents across the Full Data Intelligence Lifecyclecited by 23
  18. 2026ECHO: Terminal Agents Learn World Models for Freecited by 19
  19. 2026Recursive Synthesis for Long-Horizon Terminal Taskscited by 19
  20. 2026Quantifying Frontier LLM Capabilities for Container Sandbox Escapecited by 18
  21. 2026AgentHazard: A Benchmark for Evaluating Harmful Behavior in Computer-Use Agentscited by 18
  22. 2025ML-Dev-Bench: Comparative Analysis of AI Agents on ML development workflowscited by 17
  23. 2025BashArena: A Control Setting for Highly Privileged AI Agentscited by 17
  24. 2026CaMeLs Can Use Computers Too: System-level Security for Computer Use Agentscited by 17
  25. 2026ClawSafety: "Safe" LLMs, Unsafe Agentscited by 17
  26. 2025SWE-Dev: Evaluating and Training Autonomous Feature-Driven Software Developmentcited by 16
  27. 2025Git Context Controller: Manage the Context of LLM-based Agents like Gitcited by 15
  28. 2026A Self-Evolving Framework for Efficient Terminal Agents via Observational Context Compressioncited by 15
  29. 2026ASTRA-bench: Evaluating Tool-Use Agent Reasoning and Action Planning with Personal User Contextcited by 14
  30. 2025IRCopilot: Automated Incident Response with Large Language Modelscited by 13
  31. 2025From Reproduction to Replication: Evaluating Research Agents with Progressive Code Maskingcited by 12
  32. 2026The Blind Spot of Agent Safety: How Benign User Instructions Expose Critical Vulnerabilities in Computer-Use Agentscited by 12
  33. 2026CLI-Universe: Towards Verifiable Task Synthesis Engine for Terminal Agentscited by 12
  34. 2026SETA: Scaling Environments for Terminal Agentscited by 12
  35. 2025Secure and Efficient Access Control for Computer-Use Agents via Context Spacecited by 11
  36. 2026TerminalWorld: Benchmarking Agents on Real-World Terminal Taskscited by 10
  37. 2025SWE-MERA: A Dynamic Benchmark for Agenticly Evaluating Large Language Models on Software Engineering Taskscited by 9
  38. 2026LPS-Bench: Benchmarking Safety Awareness of Computer-Use Agents in Long-Horizon Planning under Benign and Adversarial Scenarioscited by 9

50 most cited of 78 All 78 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026Terminal-World: Scaling Terminal-Agent Environments via Agent Skills
  2. 2026Scaling Agentic Data for Long-Horizon Terminal Intelligence
  3. 2026FACET: Preserving Source Intent and Executable State in Terminal Task Synthesis
  4. 2026AgentAbstain: Do LLM Agents Know When Not to Act?
  5. 2026Long-Horizon-Terminal-Bench 2.0: Automating Diverse Task Construction and Measuring Sustained Agent Progress
  6. 2026LongCLI-Bench: A Preliminary Benchmark and Study for Long-horizon Agentic Programming in Command-Line Interfaces
  7. 2026Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verification
  8. 2026Toward Scalable Terminal Task Synthesis via Skill Graphs