Architectural Design Decisions in AI Agent Harnesses
On the map: LLM agents and tool use → General LLM agent frameworks and benchmarks
Abstract (arXiv)
AI agent systems increasingly rely on reusable non-LLM engineering infrastructure that packages tool mediation, context handling, delegation, safety control, and orchestration. Yet the architectural design decisions in this surrounding infrastructure remain understudied. This paper presents a protocol-guided, source-grounded empirical study of 70 publicly available agent-system projects, addressing three questions: which design-decision dimensions recur across projects, which co-occurrences structure those decisions, and which typical architectural patterns emerge. Methodologically, we contribute a transparent investigation procedure for analyzing heterogeneous agent-system corpora through source-code and technical-material reading. Empirically, we identify five recurring design dimensions (subagent architecture, context management, tool systems, safety mechanisms, and orchestration) and find that the corpus favors file-persistent, hybrid, and hierarchical context strategies; registry-oriented tool systems remain dominant while MCP- and plugin-oriented extensions are emerging; and intermediate isolation is common but high-assurance audit is rare. Cross-project co-occurrence analysis reveals that deeper coordination pairs with more explicit context services, stronger execution environments with more structured governance, and formalized tool-registration boundaries with broader ecosystem ambitions. We synthesize five recurring architectural patterns spanning lightweight tools, balanced CLI frameworks, multi-agent orchestrators, enterprise systems, and scenario-verticalized projects. The result provides an evidence-based account of architectural regularities in agent-system engineering, with grounded guidance for framework designers, selectors, and researchers.
Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in General LLM agent frameworks and benchmarks, one of 16 directions in LLM agents and tool use. Builds on 13 of its 28 references that are in the map. First posted on arXiv 20 Apr 2026.
Builds on
13 of 28 references.
Closest ancestors · 4
Same direction, newest first
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains17 Jun
- 2023A Survey on Large Language Model based Autonomous Agents22 Aug
- 2023AgentBench: Evaluating LLMs as Agents7 Aug
- 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Framework1 Aug
Other references · 9
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
- 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 2,006
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agentscited by 1,002
- 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 65
- 2025Model Context Protocol (MCP) at First Glance: Studying the Security and Maintainability of MCP Serverscited by 65
- 2025MCP Safety Audit: LLMs with the Model Context Protocol Allow Major Security Exploitscited by 63
Built on by
Citing papers per year: 2026: 4.
4 papers.
- 2026Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnessescited by 7
- 2026Resource Constraints and Performance in Agentic AI Systemscited by 0
- 2026Hearsay: Can an Auditor Trust the Record a Deployed Agent Harness Writes?cited by 0
- 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Taskscited by 0
- 2026VeriHarness: Scaling Agentic Verification for Long-Horizon Tasks1 Oct 2026
- 2026Hearsay: Can an Auditor Trust the Record a Deployed Agent Harness Writes?26 Sep 2026
- 2026Resource Constraints and Performance in Agentic AI Systems28 Aug 2026
- 2026Hierarchical Self-Improvement: A Framework for Task-Specific Evolvable Agent Harnesses9 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2025Control Plane as a Tool: A Scalable Design Pattern for Agentic AI Systems
- 2025AgentArch: A Comprehensive Benchmark to Evaluate Agent Architectures in Enterprise
- 2026Architecture Without Architects: How AI Coding Agents Shape Software Architecture
- 2026Architecture Matters for Multi-Agent Security
- 2025Architectures for Building Agentic AI
- 2026Architecting Agentic Communities using Design Patterns
- 2025Distinguishing Autonomous AI Agents from Collaborative Agentic Systems: A Comprehensive Framework for Understanding Modern Intelligent Architectures
- 2026AI Agents Do Not Fail Alone:The Context Fails First