Is Bash All You Need? An Empirical Study of Tool Interfaces for Enterprise Digital Worker Agents
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
In this study, we examine whether a general shell can outperform specialized tools on enterprise tasks. Shell-based agents have shown strong results in coding, but enterprise work also involves moving between applications and services, coordinating with coworkers, and performing professional analysis. We compare five tool interfaces on TheAgentCompany and APEX-Agents using Opus-4.8 and GPT-5.5: typed tools, typed tools plus bash, bash alone, bash with persistent agent-synthesized tools, and programmatic tool calling (PTC), which runs programs whose actions are restricted to a typed tool catalog. Bash alone outperforms typed tools on both benchmarks, improving score by 21.8-24.5 pp on TheAgentCompany and 4.8-7.4 pp on APEX-Agents while using 19-72% fewer total tokens. Adding typed tools or persistent tool synthesis to bash produces no detectable pooled score gain. PTC uses fewer tokens than direct typed calls with broadly similar task performance, but generally underperforms bash alone in both quality and cost efficiency. For enterprise practitioners, these results favor bash alone when arbitrary execution can be isolated and PTC when security or compliance policies require a fixed tool catalog.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 21 of its 32 references that are in the map. First posted on arXiv 10 Sep 2026.
Builds on
21 of 32 references.
Closest ancestors · 5
Same direction, newest first
- 2026LLM-as-Code: Agentic Programming for Agent Harness14 Jun
- 2025Live-SWE-agent: Can Software Engineering Agents Self-Evolve on the Fly?17 Nov
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024Agentless: Demystifying LLM-based Software Engineering Agents1 Jul
- 2024Executable Code Actions Elicit Better LLM Agents1 Feb
Other references · 16
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023Gorilla: Large Language Model Connected with Massive APIscited by 1,302
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
- 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
- 2024WorkArena: How Capable Are Web Agents at Solving Common Knowledge Work Tasks?cited by 348
- 2024TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Taskscited by 306
- 2023Large Language Models as Tool Makerscited by 293
- 2024OS-Copilot: Towards Generalist Computer Agents with Self-Improvementcited by 203
- 2023An LLM Compiler for Parallel Function Callingcited by 170
4 more
- 2024TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Taskscited by 70
- 2026APEX-Agentscited by 29
- 2025Advanced Tool Learning and Selection System (ATLASS): A Closed-Loop Framework Using LLMcited by 8
- 2026The Bitter Lesson of Tool Callingcited by 1
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026Terminal Agents Suffice for Enterprise Automation
- 2026The Devil Is in the Interface: Evaluating How Tool Architecture Shapes Coding Agent Behavior
- 2026EnterpriseClawBench: Benchmarking Agents from Real Workplace Sessions
- 2026TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents
- 2026WeaveBench: A Long-Horizon, Real-World Benchmark for Computer-Use Agents with Hybrid Interfaces
- 2026ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software
- 2026CUAWright: A Minimal Unified Interface for Digital Agents
- 2025Agent READMEs: An Empirical Study of Context Files for Agentic Coding