What Does Multi-Harness RL Learn? Credit Assignment and Portability in Coding Agents
On the map: LLM agents and tool use → Software engineering coding agents
Abstract (arXiv)
Agent reinforcement learning (RL) increasingly runs through full execution harnesses, and a multi-harness recipe mixes two choices: exposing the policy to several harnesses, and comparing their rewards inside one relative-advantage group. We isolate the second choice in repository-level coding. From one Qwen3-8B supervised warm start we replay the same frozen task-harness records from Aider, OpenHands, Qwen Code, and SWE-agent, with the same number of updates, under two rules for group-relative policy optimization (GRPO), Within (one group per task-harness pair) and Cross (harnesses pooled within a task), and score every checkpoint with a sealed SWE-bench Verified oracle on four source harnesses and a minimal harness held out of training. The evaluation harness is the dominant variable: across 24,000 sealed evaluations it moves the mean solve rate from 2.14% to 9.27%, a factor of 4.3, where the training recipe moves it by 1.16. The grouping rule is not. On the held-out harness, Cross minus Within is +0.25 pp, 95% confidence interval [-0.48, +1.02], at eight attempts per task, and +0.16 [-0.41, +0.72] pooled over three training seeds whose individual estimates change sign. Each rule's own seed range, 0.42 to 0.45 pp, exceeds the difference between them. Both rules place their largest gains on the same source harness. The pooled advantage carries the harness: an out-of-fold classifier recovers the generating harness from Cross's advantage +4.48 pp above the shuffled-label baseline and from Within's not at all, and the two rules still reach the same held-out score and action distribution inside each harness. Re-collecting half the training data on-policy does not change this. Cross-harness credit yields configuration adaptation and no more portable capability than within-harness credit. Multi-harness RL reports should state the grouping boundary and test under an unseen harness.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use. Builds on 42 of its 67 references that are in the map. First posted on arXiv 3 Sep 2026.
Builds on
42 of 67 references.
Closest ancestors · 7
Same direction, newest first
- 2026The Interplay of Harness Design and Post-Training in LLM Agents24 Jun
- 2026Harness-Bench: Measuring Harness Effects across Models in Realistic Agent Workflows27 May
- 2026Stop Comparing LLM Agents Without Disclosing the Harness7 May
- 2024Training Software Engineering Agents and Verifiers with SWE-Gym30 Dec
- 2024OpenHands: An Open Platform for AI Software Developers as Generalist Agents23 Jul
- 2024SWT-Bench: Testing and Validating Real-World Bug-Fixes with Code Agents18 Jun
- 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering6 May
Other references · 35
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2017Proximal Policy Optimization Algorithmscited by 14,441
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,146
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2023Let's Verify Step by Stepcited by 3,598
- 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
- 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
- 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
- 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
23 more
- 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
- 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
- 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
- 2021Deep Reinforcement Learning at the Edge of the Statistical Precipicecited by 781
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
- 2023Understanding the Effects of RLHF on LLM Generalisation and Diversitycited by 386
- 2023A Long Way to Go: Investigating Length Correlations in RLHFcited by 285
- 2024ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RLcited by 212
- 2021Accounting for Variance in Machine Learning Benchmarkscited by 195
- 2022Defining and Characterizing Reward Hackingcited by 184
- 2019Show Your Work: Improved Reporting of Experimental Resultscited by 164
- 2020With Little Power Comes Great Responsibilitycited by 138
- 2021The Benchmark Lotterycited by 133
- 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 130
- 2026Hindsight Credit Assignment for Long-Horizon LLM Agentscited by 51
- 2026HarnessX: A Composable, Adaptive, and Evolvable Agent Harness Foundrycited by 37
- 2026Nemotron 3 Super: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 33
- 2021Preregistering NLP Researchcited by 24
- 2026Tmax: A simple recipe for terminal agentscited by 24
- 2026Polar: Agentic RL on Any Harness at Scalecited by 12
- 2026Orchard: An Open-Source Agentic Modeling Frameworkcited by 7
- 2026OpenForgeRL: Train Harness-native Agents in Any Environmentcited by 7
Built on by
Citing papers per year: 2026: 1.
1 paper.
- 2026Harness-Zero: Harness Distillation via Agent-as-Harnesscited by 3
- 2026Harness-Zero: Harness Distillation via Agent-as-Harness21 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Cross-Benchmark Transfer from RL on Agentic Coding Tasks
- 2026LEGO-RL: Harness-Native Reinforcement Learning for Coding Agents
- 2026DiDPO: Diff-in-Diff Policy Optimization for Coding Agent Training
- 2026ClawGym II: Exploring Black-Box RL on Agent Harness
- 2026HarnessBandit: Joint Learnability-Transferability Scheduling for Multi-Harness Agentic Reinforcement Learning
- 2026Harness-RL: Black-Box Reinforcement Learning with Action-Args Decoupling for Central-Agent Multi-Agent Harnesses
- 2026Harness-R1: Learning to Edit Executable Runtime Harnesses from Agent Failure Trajectories
- 2026CHART: A Harness-Rotation Curriculum for Harness-Robust Search Agents