Self-Verified Distillation: Your Language Model Is Secretly Its Own Synthetic Data Pipeline
On the map: Synthetic data and self-improvement for LLMs → Synthetic reasoning data and bootstrapping
Abstract (arXiv)
Can post-trained large language models (LLMs) further improve themselves using only unlabeled prompts, without external teachers or feedback from tools? We study this setting starting only from unlabeled seed questions with no ground-truth solutions, across three reasoning domains: math, science, and coding. We propose Self-Verified Distillation, a simple post-training refinement algorithm in which the model generates candidate solutions to these seed questions, filters them using prompt-based self-verification, and trains on the resulting self-curated dataset. Inspired by the UQ benchmark's use of multiple validators to screen candidate answers to hard unsolved questions, we adapt this validation-based filtering idea to self-training: the model filters its own generated solutions through a three-stage cascade of cycle-consistency, factuality, and correctness checks, accepting a solution only if it passes all stages with unanimous judge votes. We find that sampling more candidate generations and using a larger verification budget during training data construction produces higher-quality self-curated data and, in turn, better reasoning models. We then train Qwen3 models at multiple scales with Self-Verified Distillation and obtain gains across all three domains. For Qwen3-4B, our method improves aggregate held-out pass@1 by +16.7 points in math (AIME26 and HMMT), +11.1 points in science (GPQA Diamond and HLE), and +8.3 points in coding (LCBv5 and LCBv6), with gains also extending to 0.6B and 8B models. Compared to our test-time-only baseline (UQ-TTC), which improves performance by spending extra compute at inference time, Self-Verified Distillation achieves better performance in most settings while requiring only a single inference call at test time.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Synthetic reasoning data and bootstrapping, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 34 of its 44 references that are in the map. First posted on arXiv 20 May 2026.
Builds on
34 of 44 references.
Closest ancestors · 3
Same direction, newest first
- 2026CHIMERA: Compact Synthetic Data for Generalizable LLM Reasoning1 Mar
- 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinking8 Jan
- 2022Large Language Models Can Self-Improve20 Oct
Other references · 31
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2023Let's Verify Step by Stepcited by 3,789
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
- 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
- 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
- 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 991
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
- 2023RAFT: Reward rAnked FineTuning for Generative Foundation Model Alignmentcited by 686
- 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
- 2025Humanity's Last Examcited by 491
19 more
- 2024ReST-MCTS*: LLM Self-Training via Process Reward Guided Tree Searchcited by 344
- 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 301
- 2026Self-Distillation Enables Continual Learningcited by 247
- 2025Learning to Reason without External Rewardscited by 210
- 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 163
- 2024C3oT: Generating Shorter Chain-of-Thought without Compromising Effectivenesscited by 119
- 2023Have LLMs Advanced Enough? A Challenging Problem Solving Benchmark For Large Language Modelscited by 77
- 2026Self-Distillation Zero: Self-Revision Turns Binary Rewards into Dense Supervisioncited by 68
- 2025Multi-Agent Evolve: LLM Self-Improve through Co-evolutioncited by 60
- 2025Reasoning to Learn from Latent Thoughtscited by 45
- 2023Benchmarking and Improving Generator-Validator Consistency of Language Modelscited by 44
- 2026Embarrassingly Simple Self-Distillation Improves Code Generationcited by 40
- 2025Retro-Search: Exploring Untaken Paths for Deeper and Efficient Reasoningcited by 34
- 2024Self-Explore: Enhancing Mathematical Reasoning in Language Models with Fine-grained Rewardscited by 31
- 2025Shrinking the Generation-Verification Gap with Weak Verifierscited by 24
- 2025Thinking Slow, Fast: Scaling Inference Compute with Distilled Reasonerscited by 22
- 2025UQ: Assessing Language Models on Unsolved Questionscited by 9
- 2026Test-time Recursive Thinking: Self-Improvement without External Feedbackcited by 7
- 2026Long-Chain Reasoning Distillation via Adaptive Prefix Alignmentcited by 6
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2025The Self-Improvement Paradox: Can Language Models Bootstrap Reasoning Capabilities without External Scaffolding?
- 2024Large Language Models Can Self-Correct with Key Condition Verification
- 2025A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning
- 2025Incentivizing LLMs to Self-Verify Their Answers
- 2026Questioning the Questions: Sustaining Self-Evolution in Reasoning Models
- 2024Think Twice Before Trusting: Self-Detection for Large Language Models through Comprehensive Answer Reflection
- 2025Self-Questioning Language Models
- 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoning