A Control System, a Dataset, and a Recipe for Making Frozen LLM Agents Learn a Domain
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
Production LLM agents are increasingly assembled from a frozen model wrapped in a harness: a prompt template, a tool set, a memory/retrieval layer, a planning strategy, and a verification policy. Two 2026 systems, Meta-Harness (Lee et al., 2026) and HyperAgents (Meta AI, 2026), show that this harness can itself be optimized or even self-rewritten by an agentic proposer -- at the cost of either an expensive code-search loop or unconstrained self-modifying code, neither of which is auditable or usable with a fully black-box model API. We take a narrower, more constrained position: treat the harness as a small, fixed, human-legible action space and learn a policy over it online with classic sample-efficient reinforcement learning (an ε-greedy contextual bandit and REINFORCE), scored against a multi-objective reward (task success, verifier score, policy compliance, cost, latency, and an unsupported-claim penalty). We instantiate this control system with DSPy (Khattab et al., 2024) as both the context assembler and the source of the strongest non-adaptive baseline (a DSPy BootstrapFewShot static prompt), and evaluate it across three verifiable task domains -- tool-use workflows, code generation (HumanEval), and multi-hop retrieval QA (HotpotQA) -- and two model providers (a local Ollama model and AWS Bedrock). We release the harness-control-system code, the cross-domain verifiable task suite, the full trajectory/reward-decomposition logs from training, and a provider-agnostic deployment recipe for applying this to a new organization's domain and verification setup.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 9 of its 11 references that are in the map. First posted on arXiv 28 Jul 2026.
Builds on
9 of 11 references.
Closest ancestors · 1
Same direction, newest first
Other references · 8
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,402
- 2023DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelinescited by 832
- 2026Meta-Harness: End-to-End Optimization of Model Harnessescited by 309
- 2026Stop Comparing LLM Agents Without Disclosing the Harnesscited by 34
- 2026A Systematic Approach to Multi-Agent AI from Advanced Regulatory Control Theory: Safe and Auditable LLM Operator Agents for Process Controlcited by 4
- 2026Stable Agentic Control: Tool-Mediated LLM Architecture for Autonomous Cyber Defensecited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Learning to Control LLM Agent Harnesses with Offline Reinforcement Learning
- 2026Context Assembly as the Controlled Variable: A Control-Theoretic View of Harness Policies for Frozen LLM Agents
- 2026EvoTrainer: Co-Evolving LLM Policies and Training Harnesses for Autonomous Agentic Reinforcement Learning
- 2026EvoHarness-RL: Learning Self-Evolving Runtime Harness for Long-Horizon LLM Agents
- 2025Learning When to Plan: Efficiently Allocating Test-Time Compute for LLM Agents
- 2026Adapting the Interface, Not the Model: Runtime Harness Adaptation for Deterministic LLM Agents
- 2025LLM-Guided Reinforcement Learning: Addressing Training Bottlenecks through Policy Modulation
- 2026Hypothesis-Driven Skill Optimization for LLM Agents