mlsanityData to 14 Oct 2026

Leveraging Knowledge Distillation for Efficient Deep Reinforcement Learning in Resource-Constrained Environments

arXiv 2310.10170 ↗First posted 16 Oct 2023cs.LG cs.AIcited by 0

On the map: Knowledge distillation → On-policy and self-distillation for LLMs

Abstract (arXiv)

This paper aims to explore the potential of combining Deep Reinforcement Learning (DRL) with Knowledge Distillation (KD) by distilling various DRL algorithms and studying their distillation effects. By doing so, the computational burden of deep models could be reduced while maintaining the performance. The primary objective is to provide a benchmark for evaluating the performance of different DRL algorithms that have been refined using KD techniques. By distilling these algorithms, the goal is to develop efficient and fast DRL models. This research is expected to provide valuable insights that can facilitate further advancements in this promising direction. By exploring the combination of DRL and KD, this work aims to promote the development of models that require fewer GPU resources, learn more quickly, and make faster decisions in complex environments. The results of this research have the capacity to significantly advance the field of DRL and pave the way for the future deployment of resource-efficient, decision-making intelligent systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in On-policy and self-distillation for LLMs, one of 12 directions in Knowledge distillation. Builds on 7 of its 13 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

7 of 13 references.

Closest ancestors · 1

Same direction, newest first

  1. 2021Catastrophic Interference in Reinforcement Learning: A Solution Based on Context Division and Knowledge Distillation1 Sep

Other references · 6

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  2. 2016OpenAI Gymcited by 1,736
  3. 2020Knowledge Distillation: A Surveycited by 1,603
  4. 2017A Brief Survey of Deep Reinforcement Learningcited by 483
  5. 2015Deep Recurrent Q-Learning for Partially Observable MDPscited by 465
  6. 2021Distilling Reinforcement Learning Tricks for Video Gamescited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Combining Trained Models in Reinforcement Learning
  2. 2025Reinforcement Learning vs. Distillation: Understanding Accuracy and Capability in LLM Reasoning
  3. 2026KD-MARL: Resource-Aware Knowledge Distillation in Multi-Agent Reinforcement Learning
  4. 2025A Practical Introduction to Deep Reinforcement Learning
  5. 2025KDRL: Post-Training Reasoning LLMs via Unified Knowledge Distillation and Reinforcement Learning
  6. 2024Efficient Exploration in Deep Reinforcement Learning: A Novel Bayesian Actor-Critic Algorithm
  7. 2024Online Policy Distillation with Decision-Attention
  8. 2021KnowRU: Knowledge Reusing via Knowledge Distillation in Multi-agent Reinforcement Learning