mlsanityData to 7 Oct 2026

Privileged Information Distillation for Language Models

arXiv 2602.04942 ↗First posted 4 Feb 2026cs.LG cs.AIcited by 79

Not on the map. Why?

Cited by 21 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Training-time privileged information (PI) can enable language models to succeed on tasks they would otherwise fail, making it a powerful tool for reinforcement learning in hard, long-horizon settings. However, transferring capabilities learned with PI to policies that must act without it at inference time remains a fundamental challenge. We study this problem in the context of distilling frontier models for multi-turn agentic environments, which typically hide their internal reasoning and expose only action trajectories. This breaks standard distillation pipelines, since successful behavior is observable, but the reasoning process is not. For this, we introduce π-Distill, a joint teacher-student objective that trains a PI-conditioned teacher and an unconditioned student simultaneously using the same model. Additionally, we also introduce On-Policy Self-Distillation (OPSD), an alternative approach that trains using Reinforcement Learning (RL) with a reverse KL-penalty between the student and the PI-conditioned teacher. We show that both of these algorithms effectively distill frontier agents using action-only PI. Specifically, we find that π-Distill and, in some cases, OPSD, outperform industry standard practices (Supervised finetuning followed by RL) that assume access to full Chain-of-Thought supervision across multiple agentic benchmarks, models, and forms of PI. We complement our results with extensive analysis that characterizes the factors enabling effective learning with PI, focusing primarily on π-Distill and characterizing when OPSD is competitive.

Key facts. Cited by 79 papers in the map (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 4 Feb 2026.

Builds on

30 of 41 references.

References · 30

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  3. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  6. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  7. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,259
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,054
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  10. 2021MuSiQue: Multihop Questions via Single-hop Question Compositioncited by 893
  11. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 840
  12. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 386
18 more
  1. 2024TravelPlanner: A Benchmark for Real-World Planning with Language Agentscited by 385
  2. 2024Quiet-STaR: Language Models Can Teach Themselves to Think Before Speakingcited by 292
  3. 2026Self-Distillation Enables Continual Learningcited by 212
  4. 2022Learning by Distilling Contextcited by 128
  5. 2023Amortizing intractable inference in large language modelscited by 94
  6. 2024Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solvingcited by 82
  7. 2025START: Self-taught Reasoner with Toolscited by 47
  8. 2023Joint Prompt Optimization of Stacked LLMs using Variational Inferencecited by 32
  9. 2025Metacognitive Reuse: Turning Recurring LLM Reasoning Into Concise Behaviorscited by 28
  10. 2025Nudging the Boundaries of LLM Reasoningcited by 25
  11. 2025Training Plug-n-Play Knowledge Modules with Deep Context Distillationcited by 22
  12. 2025RLAD: Training LLMs to Discover Abstractions for Solving Reasoning Problemscited by 21
  13. 2025How to Train Your LLM Web Agent: A Statistical Diagnosiscited by 18
  14. 2025A Comedy of Estimators: On KL Regularization in RL Training of LLMscited by 18
  15. 2025Language Models Can Learn from Verbal Feedback Without Scalar Rewardscited by 17
  16. 2025Better Estimation of the Kullback--Leibler Divergence Between Language Modelscited by 10
  17. 2023Distilling Rule-based Knowledge into Large Language Modelscited by 9
  18. 2025Variational Reasoning for Language Modelscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 79.

79 papers.

Sort:

By area:

  1. 2026Learning beyond Teacher: Generalized On-Policy Distillation with Reward Extrapolationcited by 157
  2. 2026Self-Distilled RLVRcited by 109
  3. 2026Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?cited by 87
  4. 2026Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agentscited by 75
  5. 2026HDPO: Hybrid Distillation Policy Optimization via Privileged Self-Distillationcited by 39
  6. 2026TCOD: Exploring Temporal Curriculum in On-Policy Distillation for Multi-turn Autonomous Agentscited by 38
  7. 2026Embarrassingly Simple Self-Distillation Improves Code Generationcited by 36
  8. 2026Lightning OPD: Efficient Post-Training for Large Reasoning Models with Offline On-Policy Distillationcited by 28
  9. 2026Aligning Language Models from User Interactionscited by 21
  10. 2026Unmasking On-Policy Distillation: Where It Helps, Where It Hurts, and Whycited by 21
  11. 2026D-OPSD: On-Policy Self-Distillation for Continuously Tuning Step-Distilled Diffusion Modelscited by 17
  12. 2026SOD: Step-wise On-policy Distillation for Small Language Model Agentscited by 17
38 more
  1. 2026STEPS: Selective On-Policy Self-Distillation for Reasoningcited by 15
  2. 2026StepOPSD: Step-Aware Online Preference Self-Distillation for Agent Reinforcement Learningcited by 14
  3. 2026Are Full Rollouts Necessary for On-Policy Distillation?cited by 14
  4. 2026Trajectory-Refined Distillationcited by 14
  5. 2026MAD-OPD: Breaking the Ceiling in On-Policy Distillation via Multi-Agent Debatecited by 12
  6. 2026On-Policy Self-Distillation without Any Supervisioncited by 11
  7. 2026Pass the Baton: Trajectory-Relayed On-Policy Distillationcited by 10
  8. 2026From Generic Correlation to Input-Specific Credit in On-Policy Self Distillationcited by 9
  9. 2026DOPD: Dual On-policy Distillationcited by 9
  10. 2026Multi-Rollout On-Policy Distillation via Peer Successes and Failurescited by 8
  11. 2026HERO: Hindsight-Enhanced Reflection from Environment Observations for Agentic Self-Distillationcited by 8
  12. 2026Privileged Solutions or Context-Induced Teacher Behavior? Dissecting On-Policy Self-Distillationcited by 8
  13. 2026π-Play: Multi-Agent Self-Play via Privileged Self-Distillation without External Datacited by 7
  14. 2026CEPO: RLVR Self-Distillation using Contrastive Evidence Policy Optimizationcited by 7
  15. 2026Tailoring Teaching to Aptitude: Direction-Adaptive Self-Distillation for LLM Reasoningcited by 7
  16. 2026UniSD: Towards a Unified Self-Distillation Framework for Large Language Modelscited by 6
  17. 2026Learning with Rare Success but Rich Feedback via Reflection-Enhanced Self-Distillationcited by 6
  18. 2026Learning from Language Feedback via Variational Policy Distillationcited by 6
  19. 2026Self-Distilled Policy Gradientcited by 6
  20. 2026PBSD: Privileged Bayesian Self-Distillation for Long-Horizon Credit Assignmentcited by 6
  21. 2026Self-Supervised Visual On-Policy Distillationcited by 6
  22. 2026Rethinking Privileged Information in On-Policy Self-Distillationcited by 6
  23. 2026On-Policy Self-Distillation with Sampled Demonstrations Reduces Output Diversitycited by 5
  24. 2026Structured Distillation of Web Agent Capabilities Enables Generalizationcited by 3
  25. 2026Self-Policy Distillation via Capability-Selective Subspace Projectioncited by 3
  26. 2026When Context Returns: Toward Robust Internalization in On-Policy Distillationcited by 3
  27. 2026Contrastive On-Policy Distillationcited by 3
  28. 2026The Physics of Multi-Turn Long-Horizon Planning: From Pre-training to Post-training via Single- and Multi-Teacher On-Policy Agentic Distillationcited by 3
  29. 2026Lightning OPD 2.0: Mitigating Style Bias in Cross-Teacher On-Policy Distillation for Large Reasoning Modelscited by 3
  30. 2026Teach the Magnitude, Not the Direction: Verifier-Bounded Credit Assignment for Multi-Turn Multi-step LLM Agentscited by 3
  31. 2026Beyond Absolute Imitation: Anchored Residual Guidance for Privileged On-Policy Distillationcited by 2
  32. 2026Geometric Self-Distillation for Reasoning Generalizationcited by 2
  33. 2026What Should a Self-Teacher See? Privileged Context Design for On-Policy Self-Distillationcited by 2
  34. 2026Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Modelscited by 1
  35. 2026Self-Trained Verification for Training- and Test-Time Self-Improvementcited by 1
  36. 2026Self-Distillation Policy Optimization via Visual Feedback: Bridging Code and Visual Artifactscited by 1
  37. 2026Regime-Aware Peer Specialization for Robust RAG under Heterogeneous Knowledge Conflictscited by 1
  38. 2026On-Policy Self-Distillation for Multi-Turn Image Editingcited by 1

50 most cited of 79 All 79 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026From Self-Distillation to Self-Practice: Privileged Information for Multi-Turn Agents
  2. 2026Privileged, but Biased: How PI-Conditioned Teachers Break Self-Distillation
  3. 2026β-OPSD: Deriving with Policy Optimization, Training with Self-Distillation
  4. 2026SAPD: Step-Aligned Privileged Distillation
  5. 2026dOPSD: On-Policy Self-Distillation for Diffusion Language Models
  6. 2026Scaling Properties of Same-Family On-Policy Distillation
  7. 2026The Many Faces of On-Policy Distillation: Pitfalls, Mechanisms, and Fixes
  8. 2026On-Policy Context Distillation for Language Models