Which Reasoning Trajectories Teach Students to Reason Better? A Simple Metric of Informative Alignment
On the map: LLM reasoning → Prompting and chain-of-thought reasoning
also in 1 more
Abstract (arXiv)
Long chain-of-thought (CoT) trajectories provide rich supervision signals for distilling reasoning from teacher to student LLMs. However, both prior work and our experiments show that trajectories from stronger teachers do not necessarily yield better students, highlighting the importance of data-student suitability in distillation. Existing methods assess suitability primarily through student likelihood, favoring trajectories that align closely with the student model's current behavior but overlooking more informative ones. Addressing this, we propose Rank-Surprisal Ratio (RSR), a simple metric that captures both alignment and informativeness to assess the suitability of a reasoning trajectory. RSR is motivated by the observation that effective trajectories typically balance learning signal strength and behavioral alignment by combining low absolute probability with relatively high-ranked tokens under the student model. Concretely, RSR is defined as the ratio of a trajectory's average token-wise rank to its average negative log-likelihood, and is straightforward to compute and interpret. Across five student models and reasoning trajectories from 11 diverse teachers, RSR strongly correlates with post-training reasoning performance (average Spearman 0.86), consistently outperforming existing metrics. We further demonstrate its practical utility in both trajectory selection and teacher selection.
Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 48 of its 70 references that are in the map. First posted on arXiv 20 Jan 2026.
Builds on
48 of 70 references.
Closest ancestors · 10
Same direction, newest first
- 2025Beyond Scaling Law: A Data-Efficient Distillation Framework for Reasoning13 Aug
- 2025MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants2 Jul
- 2025Through the Valley: Path to Effective Long CoT Training for Small Language Models9 Jun
- 2025What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning28 May
- 2025Select2Reason: Efficient Instruction-Tuning Data Selection for Long-CoT Reasoning22 May
- 2025Rethinking the Generation of High-Quality CoT Data from the Perspective of LLM-Adaptive Question Difficulty Grading16 Apr
- 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoning25 Feb
- 2025Small Models Struggle to Learn from Strong Reasoners17 Feb
- 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!11 Feb
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan
Other references · 38
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
- 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
- 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
- 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
- 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
- 2025s1: Simple test-time scalingcited by 1,341
- 2023LIMA: Less Is More for Alignmentcited by 1,100
- 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
- 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
26 more
- 2023Instruction Tuning with GPT-4cited by 686
- 2019Improved Knowledge Distillation via Teacher Assistantcited by 573
- 2025LIMO: Less is More for Reasoningcited by 498
- 2023Studying Large Language Model Generalization with Influence Functionscited by 326
- 2023From Quantity to Quality: Boosting LLM Performance with Self-Guided Data Selection for Instruction Tuningcited by 307
- 2025OpenThoughts: Data Recipes for Reasoning Modelscited by 304
- 2025Llama-Nemotron: Efficient Reasoning Modelscited by 158
- 2025Phi-4-reasoning Technical Reportcited by 151
- 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 127
- 2025Learning What Reinforcement Learning Can't: Interleaved Online Fine-Tuning for Hardest Questionscited by 97
- 2025Retaining by Doing: The Role of On-Policy Data in Mitigating Forgettingcited by 94
- 2024Speculative Knowledge Distillation: Bridging the Teacher-Student Gap Through Interleaved Samplingcited by 90
- 2025BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoningcited by 63
- 2024Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuningcited by 57
- 2025The Best Instruction-Tuning Data are Those That Fitcited by 52
- 2025ReasonFlux-PRM: Trajectory-Aware PRMs for Long Chain-of-Thought Reasoning in LLMscited by 46
- 2023Towards the Law of Capacity Gap in Distilling Language Modelscited by 44
- 2025ReCode: Reinforcing Code Generation with Reasoning-Process Rewardscited by 25
- 2025Measuring Data Diversity for Instruction Tuning: A Systematic Analysis and A Reliable Metriccited by 24
- 2025Influence Functions for Efficient Data Selection in Reasoningcited by 10
- 2025Information-Guided Identification of Training Data Imprint in (Proprietary) Large Language Modelscited by 8
- 2025The Signal is in the Steps: Local Scoring for Reasoning Data Selectioncited by 8
- 2025Analyzing the Effects of Supervised Fine-Tuning on Model Knowledge from Token and Parameter Levelscited by 7
- 2025In Good GRACEs: Principled Teacher Selection for Knowledge Distillationcited by 5
- 2025Warm Up Before You Train: Unlocking General Reasoning in Resource-Constrained Settingscited by 4
- 2025AIR: Post-training Data Selection for Reasoning via Attention Head Influencecited by 3
Built on by
Citing papers per year: 2026: 13.
13 papers.
Sort:
By area:
- 2026Towards Understanding On-Policy Distillation through the Lens of Test-Time Scalingcited by 5
- 2026On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuningcited by 4
- 2026Execution-First Synthetic Tool-Use Trace Generation for LLM Agentscited by 2
- 2026DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillationcited by 1
- 2026When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selectioncited by 1
- 2026LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillationcited by 1
- 2026Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Tracescited by 0
- 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillationcited by 0
- 2026Which Medical Questions Deserve Rationales? Perturbation-Sensitive Selection for Robust QAcited by 0
- 2026On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Trainingcited by 0
- 2026Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Trainingcited by 0
- 2026Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Modelscited by 0
1 more
- 2026Cowork-Gym: Synthesizing Interactive Work Environments at Scale13 Oct 2026
- 2026Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Models12 Oct 2026
- 2026Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training9 Oct 2026
- 2026On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training29 Sep 2026
- 2026Which Medical Questions Deserve Rationales? Perturbation-Sensitive Selection for Robust QA9 Sep 2026
- 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation30 Aug 2026
- 2026Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling12 Aug 2026
- 2026Execution-First Synthetic Tool-Use Trace Generation for LLM Agents31 Jul 2026
- 2026LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation28 May 2026
- 2026Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Traces28 May 2026
- 2026When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection26 May 2026
- 2026DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation15 May 2026
- 2026Towards Understanding On-Policy Distillation through the Lens of Test-Time Scalingcited by 5
- 2026On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuningcited by 4
- 2026DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillationcited by 1
- 2026When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selectioncited by 1
- 2026LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillationcited by 1
- 2026Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Tracescited by 0
- 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillationcited by 0
- 2026Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Trainingcited by 0
- 2026Learning More by Training Less: Uncertainty-Guided Progressive Contraction for Efficient Reasoning Post-Training9 Oct 2026
- 2026MI-Distillation: Selecting from Model-Interpolated Instruct-Reasoning Data Spectrum for Chain-of-Thought Distillation30 Aug 2026
- 2026Towards Understanding On-Policy Distillation through the Lens of Test-Time Scaling12 Aug 2026
- 2026LARK: Learnability-Grounded Trajectory Selection for Efficient Reasoning Distillation28 May 2026
- 2026Diagnosing Harmful Continuation in Answer-Correct Long-CoT Training Traces28 May 2026
- 2026When the Strongest Teacher Is Not the Best Teacher: Student-Centric Answer Selection26 May 2026
- 2026DeltaPrompts: Escaping the Zero-Delta Trap in Multimodal Distillation15 May 2026
- 2026On the Role of Reasoning Patterns in the Generalization Discrepancy of Long Chain-of-Thought Supervised Fine-Tuning2 Apr 2026
- 2026Which Medical Questions Deserve Rationales? Perturbation-Sensitive Selection for Robust QAcited by 0
- 2026On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Trainingcited by 0
- 2026Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Modelscited by 0
- 2026Cowork-Gym: Synthesizing Interactive Work Environments at Scalecited by 0
- 2026Cowork-Gym: Synthesizing Interactive Work Environments at Scale13 Oct 2026
- 2026Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Models12 Oct 2026
- 2026On-Policy Parameter Update Direction Underlies Generalization in LLM Post-Training29 Sep 2026
- 2026Which Medical Questions Deserve Rationales? Perturbation-Sensitive Selection for Robust QA9 Sep 2026
Similar, not cited
Close in wording, no citation either way
- 2026Long-Chain Reasoning Distillation via Adaptive Prefix Alignment
- 2026Beyond Teacher Likelihood: Group-Calibrated On-Policy Distillation for Long-Context Reasoning
- 2026Beyond Imitation: Auditing the Recoverability of Reasoning in Distilled Models
- 2026Student-in-the-Loop Chain-of-Thought Distillation via Generation-Time Selection
- 2026Expected Reasoning-Step Return Unifies On-Policy Learning from Rewards and Teachers
- 2026Validity-Calibrated Reasoning Distillation
- 2026Distribution-Aligned Sequence Distillation for Superior Long-CoT Reasoning
- 2026TeacherGRPO: Closing the Capacity Gap in Reasoning Distillation via Teacher Alignment