# Training with Harnesses: On-Policy Harness Self-Distillation for Complex Reasoning

arXiv: https://arxiv.org/abs/2605.08741 · HTML: https://mlsanity.com/paper/2605.08741/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 9 May 2026.

## Placed in

- LLM reasoning / [Prompting and chain-of-thought reasoning](https://mlsanity.com/lr/prompting-and-chain-of-thought-reasoning/)
- Knowledge distillation / [On-policy and self-distillation for LLMs](https://mlsanity.com/distill/on-policy-and-self-distillation-for-llms/)

## Builds on (most cited first, 23 of 23)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes](https://mlsanity.com/paper/2306.13649/) (2023)
- [GLM-5: from Vibe Coding to Agentic Engineering](https://mlsanity.com/paper/2602.15763/) (2026)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models](https://mlsanity.com/paper/2601.18734/) (2026)
- [Reinforcement Learning via Self-Distillation](https://mlsanity.com/paper/2601.20802/) (2026)
- [Meta-Harness: End-to-End Optimization of Model Harnesses](https://mlsanity.com/paper/2603.28052/) (2026)
- [DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning](https://mlsanity.com/paper/2504.11456/) (2025)
- [Self-Distillation Enables Continual Learning](https://mlsanity.com/paper/2601.19897/) (2026)
- [MiMo-V2-Flash Technical Report](https://mlsanity.com/paper/2601.02780/) (2026)
- [CAIL2018: A Large-Scale Legal Dataset for Judgment Prediction](https://mlsanity.com/paper/1807.02478/) (2018)
- [A Survey of On-Policy Distillation for Large Language Models](https://mlsanity.com/paper/2604.00626/) (2026)
- [Revisiting On-Policy Distillation: Empirical Failure Modes and Simple Fixes](https://mlsanity.com/paper/2603.25562/) (2026)
- [Self-Distilled RLVR](https://mlsanity.com/paper/2604.03128/) (2026)
- [Scaling Reasoning Efficiently via Relaxed On-Policy Distillation](https://mlsanity.com/paper/2603.11137/) (2026)
- [AutoHarness: improving LLM agents by automatically synthesizing a code harness](https://mlsanity.com/paper/2603.03329/) (2026)
- [Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervision](https://mlsanity.com/paper/2604.12002/) (2026)
- [Natural-Language Agent Harnesses](https://mlsanity.com/paper/2603.25723/) (2026)
- [CRISP: Compressed Reasoning via Iterative Self-Policy Distillation](https://mlsanity.com/paper/2603.05433/) (2026)
- [MAGDi: Structured Distillation of Multi-Agent Interaction Graphs Improves Reasoning in Smaller Language Models](https://mlsanity.com/paper/2402.01620/) (2024)
- [Step 3.5 Flash: Open Frontier-Level Intelligence with 11B Active Parameters](https://mlsanity.com/paper/2602.10604/) (2026)
- [Online Experiential Learning for Language Models](https://mlsanity.com/paper/2603.16856/) (2026)

## Built on by (most cited first, 10 of 10)

- [Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning](https://mlsanity.com/paper/2608.01589/) (2026)
- [Harness-Zero: Harness Distillation via Agent-as-Harness](https://mlsanity.com/paper/2609.24974/) (2026)
- [Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models](https://mlsanity.com/paper/2605.30251/) (2026)
- [When Context Returns: Toward Robust Internalization in On-Policy Distillation](https://mlsanity.com/paper/2606.11627/) (2026)
- [Agent Harness Distillation: Inference-Time Harness Extraction and Exploitation in Autonomous Multi-Agent Systems](https://mlsanity.com/paper/2607.28147/) (2026)
- [SR-OPSD: Self-Referenced On-Policy Self-Distillation](https://mlsanity.com/paper/2608.09745/) (2026)
- [Harness-Aware Distillation for Small Language Model Agents](https://mlsanity.com/paper/2610.02858/) (2026)
- [PreSO: Privileged Representation Isolation for On-Policy Self-Distillation](https://mlsanity.com/paper/2610.14001/) (2026)
- [Does self-distillation improve reasoning?](https://mlsanity.com/paper/2610.18613/) (2026)
- [Harness Policy Distillation: Aligning Agent Policies with Runtime Semantics](https://mlsanity.com/paper/2610.20261/) (2026)

## Abstract (arXiv)

Inference-time harnesses substantially improve large language models on complex reasoning tasks. However, the intrinsic capabilities of the underlying model remain unchanged by the addition of these external workflows. To bridge this gap, we introduce On-Policy Harness Self-Distillation (OPHSD), which employs the harness-augmented current model as a teacher for self-distillation, thereby introducing extra supervisory signals from the harness beyond training data. OPHSD internalizes task-specific harness capabilities into the student model, yielding robust generalizability and strong standalone performance across diverse reasoning tasks. Evaluated across draft--verify harness for text classification and plan--solve for mathematical reasoning tasks, OPHSD consistently outperforms strong baselines (e.g., +10.83% over OPSD on HMMT25). Our analysis further indicates that reattaching the harness during inference yields no additional benefits and can even degrade performance, suggesting that complex harnesses need not always be permanent fixtures; instead, they can serve as temporary training scaffolds whose benefits are permanently fed back into the base model. Our code and training data are available at https://github.com/zzy1127/OPHSD-On-Policy-Harness-Self-Distillation.