mlsanityData to 7 Oct 2026

From Prompts to Contracts: Harness Engineering for Auditable Enterprise LLM Agents

arXiv 2607.08028 ↗First posted 9 Jul 2026cs.AI cs.CL cs.SEcited by 3

On the map: LLM agents and tool use → Software engineering coding agents

Abstract (arXiv)

Enterprise large language model (LLM) applications often begin as prototypes whose behavior is carried by prompts and retrieval context. Productization adds requirements for source boundaries, entity routing, answer contracts, and reproducible traces. We present a harness-engineering approach that reconstructs this pattern into a traceable, auditable LLM-agent architecture: deterministic behavior moves into code, manifests, schemas, and validation artifacts around a replaceable composition boundary, while source-backed claims remain the authority for runtime answers. We instantiate it on a public-data slice of five Korean corporate groups (25 listed companies) and evaluate three research questions. (1) The harness preserves its source-grounding, entity-routing, trace, output-hygiene, and recommendation-language contracts across the fixed validation scenarios; a fault-injection control confirms the validators flag deliberately broken contracts. (2) The checks the harness enforces held under model substitution: across three hosted models, they passed on all 270 composition-boundary runs; failures were confined to the model-composed side and were caught and recorded. (3) The code-owned guarantees are load-bearing, not reproducible by prompting alone: holding the model fixed and varying only the enforcement layer, prompt instructions alone let recommendation-language and internal-trace-leakage violations reach the reader, which the harness blocks entirely. A bolt-on external guardrail prevents such violations too but over-refuses, dropping utility to 88/120 where the harness preserves full utility (120/120); in this ablation, only code-owned enforcement preserves both safety and utility. The result is a reusable engineering pattern for turning exploratory prototypes into auditable applications with versioned source, control, and validation artifacts.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 40 of its 62 references that are in the map. First posted on arXiv 9 Jul 2026.

Builds on

40 of 62 references.

Closest ancestors · 5

Same direction, newest first

  1. 2026Code as Agent Harness18 May
  2. 2026AI Harness Engineering: A Runtime Substrate for Foundation-Model Software Agents13 May
  3. 2026Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses28 Apr
  4. 2025Why Do Multi-Agent LLM Systems Fail?17 Mar
  5. 2023AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation16 Aug

Other references · 35

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,160
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 2,950
  7. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,222
  8. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,203
  9. 2023Retrieval-Augmented Generation for Large Language Models: A Surveycited by 2,185
  10. 2022Survey of Hallucination in Natural Language Generationcited by 2,056
  11. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  12. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,800
23 more
  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  2. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  3. 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 803
  4. 2023Enabling Large Language Models to Generate Text with Citationscited by 584
  5. 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
  6. 2023Ragas: Automated Evaluation of Retrieval Augmented Generationcited by 408
  7. 2023NeMo Guardrails: A Toolkit for Controllable and Safe LLM Applications with Programmable Railscited by 347
  8. 2022Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLPcited by 316
  9. 2023Evaluating Verifiability in Generative Search Enginescited by 297
  10. 2023ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systemscited by 259
  11. 2021Measuring Attribution in Natural Language Generation Modelscited by 253
  12. 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 248
  13. 2025Evaluation and Benchmarking of LLM Agents: A Surveycited by 145
  14. 2023Auditing large language models: a three-layered approachcited by 135
  15. 2022Machine Learning Operations (MLOps): Overview, Definition, and Architecturecited by 126
  16. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  17. 2021Software Engineering for AI-Based Systems: A Surveycited by 68
  18. 2026Externalization in LLM Agents: A Unified Review of Memory, Skills, Protocols and Harness Engineeringcited by 68
  19. 2024PersonaRAG: Enhancing Retrieval-Augmented Generation Systems with User-Centric Agentscited by 28
  20. 2025Eliza: A Web3 friendly AI Agent Operating Systemcited by 14
  21. 2025Vibe Coding as a Reconfiguration of Intent Mediation in Software Development: Definition, Implications, and Research Agendacited by 13
  22. 2026Retrieval as Reasoning: Self-Evolving Agent-Native Retrieval via LLM-Wikicited by 9
  23. 2026WiCER: Wiki-memory Compile, Evaluate, Refine Iterative Knowledge Compilation for LLM Wiki Systemscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 3.

3 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Better Harnesses, Smaller Models: Building 90% Cheaper Agents via Automated Harness Adaptation
  2. 2026Iterative Audit Convergence in LLM-Managed Multi-Agent Systems: A Case Study in Prompt-Engineering Quality Assurance
  3. 2026Harness-MU: A Safe, Governed, and Effective Harness for Multi-User LLM Agents
  4. 2026Who Tests the Testers? Systematic Enumeration and Coverage Audit of LLM Agent Tool Call Safety
  5. 2026Persona-Execution Separation: An Architecture Pattern for Evolving LLM Agents under Execution Audit
  6. 2026Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
  7. 2026A Large-Scale Empirical Study of Quality Assurance Practices and Gaps in AI Agents
  8. 2026Grow the Harness, Not the Context: From Strategy-Free Scaffolds to Reusable Specialist Agents