mlsanityData to 14 Oct 2026

Teacher-Free Self-Training Amplifies but Does Not Compound: A Pass@K Crossover on a Free-Verifier Domain

arXiv 2606.07856 ↗First posted 5 Jun 2026cs.LGcited by 4

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

When a language model trains on its own verified outputs, does it acquire capability beyond its base, or merely get better at expressing capability the base already had? We make the question decidable with a teacher-free "constellation" -- a generator, a learned critic, and a free exact verifier -- on a FlashFill-style "trapdoor" DSL, where verified (problem, solution) pairs are cheap to synthesize, hard to invert, and free to check exactly. Everything runs on one 4-bit Qwen3-4B on a single 24 GB GPU, with no model in the loop larger than the base. We report three findings. (i) Critic-guided selection beats verifier-filtered best-of-k by +9.1 pp (6/6 seeds), with the entire gain localized to tasks where candidates disagree on held-out inputs. (ii) Per-round STaR self-training raises the ceiling but never accelerates -- the gain tracks remaining headroom and decelerates across K=4 independent training trajectories. (iii) The domain has no clean zero-capability frontier, so the usual "0% → climb = emergence" test is invalid here. A measured pass@K crossover settles the diagnosis: the trained model wins at the operating budget (pass@8) but the base overtakes it at a large budget (pass@64) on every trajectory, so self-training concentrates probability mass rather than expanding reach. This is amplification, not compounding. (K=4 is indicative, not yet a robust across-trajectory CI.)

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 10 of its 14 references that are in the map. First posted on arXiv 5 Jun 2026.

Builds on

10 of 14 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Reinforced Self-Training (ReST) for Language Modeling17 Aug

Other references · 9

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  3. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  4. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Modelscited by 427
  5. 2025ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries in Large Language Modelscited by 194
  6. 2025Rewarding the Unlikely: Lifting GRPO Beyond Distribution Sharpeningcited by 80
  7. 2025Jointly Reinforcing Diversity and Quality in Language Model Generationscited by 80
  8. 2025Weight Ensembling Improves Reasoning in Language Modelscited by 38
  9. 2025Differential Smoothing Mitigates Sharpening and Improves LLM Reasoningcited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026The Verifier is the Curriculum: Precision Sets the Return on Search in Code Self-Distillationcited by 3
  2. 2026Phantom Gains: Auditing Self-Improvement Against a Measured Nullcited by 2
  3. 2026When Correct Solutions Repeat: Rarity-Aware Credit Redistribution for GRPOcited by 1
  4. 2026RLVR is a Kernel, Not a Function: Statistical Inference for pass@k Crossoverscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026A Training-Free Regeneration Paradigm: Contrastive Reflection Memory Guided Self-Verification and Self-Improvement
  2. 2026Introspective X Training: Feedback Conditioning Improves Scaling Across all LLM Training Stages
  3. 2026Training-Free Inference-Time Self-Reflection and Cost-Bounded Early Stopping for Large Language Models
  4. 2026Proof-Carrying Cognition: Closing the Verification Gap with Reality-Settled Reward
  5. 2026Self-Policy Distillation via Capability-Selective Subspace Projection
  6. 2026Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR
  7. 2026Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training
  8. 2026Improving Few-Step Language Flows with Untied Self-Conditioning