From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 40 of its 62 references that are in the map. First posted on arXiv 9 Jul 2026.
Builds on
40 of 62 references.
Closest ancestors · 5
Same direction, newest first
- 2026Code as Agent Harness18 May
- 2026AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents13 May
- 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
- 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
- 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug
Other references · 35
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
- 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
- 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
- 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
- 2022Survey of Hallucination in Natural Language Generationcited by 2,056
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
- 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
23 more
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
- 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
- 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
- 2023Evaluating Verifiability in Generative Search Enginescited by 297
- 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
- 2021Measuring Attribution in Natural Language Generation Modelscited by 253
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
- 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
- 2023Auditing large language models: a three-layered approachcited by 135
- 2022Machine Learning Operations (MLOps): Overview, Definition, and Architecturecited by 126
- 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
- 2021Software Engineering for AI-Based Systems: A Surveycited by 68
- 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
- 2024PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agentscited by 28
- 2025Eliza: A Web3 friendly AI Agent Operating Systemcited by 14
- 2025Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agendacited by 13
- 2026Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wikicited by 9
- 2026WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systemscited by 4
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligencecited by 1
- 2026Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scalecited by 0
- 2026Cross-Provider Review as a Runtime Contract for Coding Agents: A Controlled Pilot and Fault-Injection Studycited by 0
- 2026Cross-Provider Review as a Runtime Contract for Coding Agents: A Controlled Pilot and Fault-Injection Study7 Oct 2026
- 2026Verify, Don't Trust: Agentic Model Development for Video Discovery Retrieval at Scale18 Sep 2026
- 2026Graph Engineering in the Era of LLM Agents: From Individual Intelligence to System Intelligence21 Aug 2026
Similar, not cited
Close in wording, no citation either way
- 2026Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation
- 2026Iterative Audit Convergence in LLM-Managed Multi-Agent Systems: A Case Study in Prompt-Engineering Quality Assurance
- 2026Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agents
- 2026Who Tests the Testers? Systematic Enumeration and Coverage Audit of LLM Agent Tool Call Safety
- 2026Persona-Execution Separation: An Architecture Pattern for Evolving LLM Agents under Execution Audit
- 2026Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
- 2026A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents
- 2026Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents