mlsanityData to 7 Oct 2026

Distilling Policy Distillation

arXiv 1902.02186 ↗First posted 6 Feb 2019cs.LG cs.AI stat.MLcited by 81

Not on the map. Why?

Abstract (arXiv)

The transfer of knowledge from one policy to another is an important tool in Deep Reinforcement Learning. This process, referred to as distillation, has been used to great success, for example, by enhancing the optimisation of agents, leading to stronger performance faster, on harder domains [26, 32, 5, 8]. Despite the widespread use and conceptual simplicity of distillation, many different formulations are used in practice, and the subtle variations between them can often drastically change the performance and the resulting objective that is being optimised. In this work, we rigorously explore the entire landscape of policy distillation, comparing the motivations and strengths of each variant through theoretical and empirical analysis. Our results point to three distillation techniques, that are preferred depending on specifics of the task. Specifically a newly proposed expected entropy regularised distillation allows for quicker learning in a wide range of situations, while still guaranteeing convergence.

Key facts. Cited by 81 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Feb 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 12; 2022: 13; 2023: 12; 2024: 18; 2025: 12; 2026: 14.

81 papers.

Sort:

By area:

  1. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 177
  2. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 136
  3. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  4. 2021Spectral Normalisation for Deep Reinforcement Learning: an Optimisation Perspectivecited by 64
  5. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 63
  6. 2024Rethinking Kullback-Leibler Divergence in Knowledge Distillation for Large Language Modelscited by 60
  7. 2021Robot Learning from Randomized Simulations: A Reviewcited by 56
  8. 2024Sparse Diffusion Policy: A Sparse, Reusable, and Flexible Policy for Robot Learningcited by 56
  9. 2022Retrieval-Augmented Reinforcement Learningcited by 53
  10. 2021SECANT: Self-Expert Cloning for Zero-Shot Generalization of Visual Policiescited by 42
  11. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 35
  12. 2022Honor of Kings Arena: an Environment for Generalization in Competitive Reinforcement Learningcited by 27
38 more
  1. 2024MiniPLM: Knowledge Distillation for Pre-Training Language Modelscited by 27
  2. 2026Demystifying OPD: Length Inflation and Stabilization Strategies for Large Language Modelscited by 24
  3. 2025Refined Policy Distillation: From VLA Generalists to RL Expertscited by 19
  4. 2023Omnidirectional Information Gathering for Knowledge Transfer-based Audio-Visual Navigationcited by 18
  5. 2023TWIST: Teacher-Student World Model Distillation for Efficient Sim-to-Real Transfercited by 18
  6. 2023TGRL: An Algorithm for Teacher Guided Reinforcement Learningcited by 16
  7. 2023Guided Online Distillation: Promoting Safe Reinforcement Learning by Offline Demonstrationcited by 14
  8. 2022A System for Morphology-Task Generalization via Unified Representation and Behavior Distillationcited by 12
  9. 2022Learning Dynamics and Generalization in Reinforcement Learningcited by 11
  10. 2024Distilling Morphology-Conditioned Hypernetworks for Efficient Universal Morphology Controlcited by 11
  11. 2025Iterative Distillation for Reward-Guided Fine-Tuning of Diffusion Models in Biomolecular Designcited by 10
  12. 2024Proximal Policy Distillationcited by 9
  13. 2022Efficient Policy Space Response Oraclescited by 8
  14. 2021Human-Inspired Multi-Agent Navigation using Knowledge Distillationcited by 7
  15. 2022Hierarchical Kickstarting for Skill Transfer in Reinforcement Learningcited by 7
  16. 2022Cyclic Policy Distillation: Sample-Efficient Sim-to-Real Reinforcement Learning with Domain Randomizationcited by 7
  17. 2026RAPO: Expanding Exploration for LLM Agents via Retrieval-Augmented Policy Optimizationcited by 6
  18. 2021Neural-to-Tree Policy Distillation with Policy Improvement Criterioncited by 5
  19. 2022Reinforcement Learning in Practice: Opportunities and Challengescited by 5
  20. 2022Multi-Source Transfer Learning for Deep Model-Based Reinforcement Learningcited by 5
  21. 2024Dataset Distillation for Offline Reinforcement Learningcited by 5
  22. 2025Contextual Thompson Sampling via Generation of Missing Datacited by 5
  23. 2021Pattern Transfer Learning for Reinforcement Learning in Order Dispatchingcited by 4
  24. 2023Factual and Personalized Recommendations using Language Models and Reinforcement Learningcited by 4
  25. 2024Leveraging Constraint Programming in a Deep Learning Approach for Dynamically Solving the Flexible Job-Shop Scheduling Problemcited by 4
  26. 2024Interpretable end-to-end Neurosymbolic Reinforcement Learning agentscited by 4
  27. 2025Advantage-Guided Distillation for Preference Alignment in Small Language Modelscited by 4
  28. 2021Distilled Domain Randomizationcited by 3
  29. 2024Grasp Anything: Combining Teacher-Augmented Policy Gradient Learning with Instance Segmentation to Grasp Arbitrary Objectscited by 3
  30. 2024Learning to Build by Building Your Own Instructionscited by 3
  31. 2024Online Control-Informed Learningcited by 3
  32. 2025Guided Policy Optimization under Partial Observabilitycited by 3
  33. 2022Policy Distillation with Selective Input Gradient Regularization for Efficient Interpretabilitycited by 2
  34. 2022Soft Action Priors: Towards Robust Policy Transfercited by 2
  35. 2023Introspective Action Advising for Interpretable Transfer Learningcited by 2
  36. 2025From Abstraction to Reality: DARPA's Vision for Robust Sim-to-Real Autonomycited by 2
  37. 2025Enhancing Security in Deep Reinforcement Learning: A Comprehensive Survey on Adversarial Attacks and Defensescited by 2
  38. 2021Offline RL With Resource Constrained Online Deploymentcited by 1

50 most cited of 81 All 81 (JSON)