# Same Model, Different Harness: Different Coding-Agent Results

arXiv: https://arxiv.org/abs/2608.26218 · HTML: https://mlsanity.com/paper/2608.26218/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Software engineering coding agents, one of 16 directions in LLM agents and tool use.
- Builds on 3 of its 8 references that are in the map.
- First posted on arXiv 26 Aug 2026.

## Placed in

- LLM agents and tool use / [Software engineering coding agents](https://mlsanity.com/agents/software-engineering-coding-agents/)

## Builds on (most cited first, 3 of 3)

- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Nemotron-Cascade 2: Post-Training LLMs with Cascade RL and Multi-Domain On-Policy Distillation](https://mlsanity.com/paper/2603.19220/) (2026)
- [FeatureBench: Benchmarking Agentic Coding for Complex Feature Development](https://mlsanity.com/paper/2602.10975/) (2026)

## Built on by (most cited first, 4 of 4)

- [An Empirical Study of Harness Design for Coding Agents](https://mlsanity.com/paper/2609.20804/) (2026)
- [Harness or Model? Isolating the Harness Effect in Agentic Coding with a Contamination-Controlled Private Suite](https://mlsanity.com/paper/2609.11987/) (2026)
- [Identical Runs, Different Results: Benchmarking AI Coding Agents on Open-Weight Models](https://mlsanity.com/paper/2609.33812/) (2026)
- [What Does a Harness Buy? Tokens, Mostly](https://mlsanity.com/paper/2610.04433/) (2026)

## Abstract (arXiv)

A coding agent combines a model with a harness, which decides what the model sees, which tools it can use, and how the work continues. We ask whether changing the harness changes the result when the model and task stay fixed. We compare two configurations of the same harness on three coding benchmarks. The control supplies the full conversation in time order, while the treatment keeps the same record but mechanically shortens older tool results as the context fills and responds to repeated or stalled work. Under tight context, the treatment raises mean per-task fail-to-pass fraction (F2PF) in all three pressure comparisons and increases complete solutions on SWE-bench Verified and SWE-bench Pro. The tight-window Verified comparison uses 169 tasks, a 20,480-token window, and a fixed 480-second attempt endpoint; on this cohort, treatment raises mean per-task F2PF from 28 percent to 49 percent and complete solutions from 43 to 72. Without model-specific retuning, the same frozen treatment also raises both endpoints on the same cohort for three additional models with different designs. In the wide-window Qwen3.6 comparisons, observed arm outcomes are close on Verified and Pro, while FeatureBench retains a higher mean per-task F2PF under treatment. On the wide-window Verified cohort, treatment also serves fewer prompt tokens per turn. Because changing the harness changed what unchanged model weights could accomplish, coding-agent evaluations should treat the model and harness together as the tested solver.