# Teacher-Free Self-Training Amplifies but Does Not Compound: A Pass@K Crossover on a Free-Verifier Domain

arXiv: https://arxiv.org/abs/2606.07856 · HTML: https://mlsanity.com/paper/2606.07856/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs.
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 5 Jun 2026.

## Placed in

- Synthetic data and self-improvement for LLMs / [Self-rewarding alignment and self-training](https://mlsanity.com/synth/self-rewarding-alignment-and-self-training/)

## Builds on (most cited first, 10 of 10)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?](https://mlsanity.com/paper/2504.13837/) (2025)
- [STaR: Bootstrapping Reasoning With Reasoning](https://mlsanity.com/paper/2203.14465/) (2022)
- [Reinforced Self-Training (ReST) for Language Modeling](https://mlsanity.com/paper/2308.08998/) (2023)
- [Scaling Relationship on Learning Mathematical Reasoning with Large Language Models](https://mlsanity.com/paper/2308.01825/) (2023)
- [ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Models](https://mlsanity.com/paper/2505.24864/) (2025)
- [Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpening](https://mlsanity.com/paper/2506.02355/) (2025)
- [Jointly Reinforcing Diversity and Quality in Language Model Generations](https://mlsanity.com/paper/2509.02534/) (2025)
- [Weight Ensembling Improves Reasoning in Language Models](https://mlsanity.com/paper/2504.10478/) (2025)
- [Differential Smoothing Mitigates Sharpening and Improves LLM Reasoning](https://mlsanity.com/paper/2511.19942/) (2025)

## Built on by (most cited first, 4 of 4)

- [The Verifier is the Curriculum: Precision Sets the Return on Search in Code Self-Distillation](https://mlsanity.com/paper/2607.09709/) (2026)
- [Phantom Gains: Auditing Self-Improvement Against a Measured Null](https://mlsanity.com/paper/2608.20290/) (2026)
- [When Correct Solutions Repeat: Rarity-Aware Credit Redistribution for GRPO](https://mlsanity.com/paper/2608.03467/) (2026)
- [RLVR is a Kernel, Not a Function: Statistical Inference for pass@k Crossovers](https://mlsanity.com/paper/2609.22547/) (2026)

## Abstract (arXiv)

When a language model trains on its own verified outputs, does it acquire capability beyond its base, or merely get better at expressing capability the base already had? We make the question decidable with a teacher-free "constellation" -- a generator, a learned critic, and a free exact verifier -- on a FlashFill-style "trapdoor" DSL, where verified (problem, solution) pairs are cheap to synthesize, hard to invert, and free to check exactly. Everything runs on one 4-bit Qwen3-4B on a single 24 GB GPU, with no model in the loop larger than the base. We report three findings. (i) Critic-guided selection beats verifier-filtered best-of-k by +9.1 pp (6/6 seeds), with the entire gain localized to tasks where candidates disagree on held-out inputs. (ii) Per-round STaR self-training raises the ceiling but never accelerates -- the gain tracks remaining headroom and decelerates across K=4 independent training trajectories. (iii) The domain has no clean zero-capability frontier, so the usual "0% → climb = emergence" test is invalid here. A measured pass@K crossover settles the diagnosis: the trained model wins at the operating budget (pass@8) but the base overtakes it at a large budget (pass@64) on every trajectory, so self-training concentrates probability mass rather than expanding reach. This is amplification, not compounding. (K=4 is indicative, not yet a robust across-trajectory CI.)