mlsanityData to 14 Oct 2026

Kickstarting Deep Reinforcement Learning

arXiv 1803.03835 ↗First posted 10 Mar 2018cs.LGcited by 81

Not on the map. Why?

Cited by 21 in Deep reinforcement learning methods, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a method for using previously-trained 'teacher' agents to kickstart the training of a new 'student' agent. To this end, we leverage ideas from policy distillation and population based training. Our method places no constraints on the architecture of the teacher or student agents, and it regulates itself to allow the students to surpass their teachers in performance. We show that, on a challenging and computationally-intensive multi-task benchmark (DMLab-30), kickstarted training improves the data efficiency of new agents, making it significantly easier to iterate on their design. We also show that the same kickstarting pipeline can allow a single student agent to leverage multiple 'expert' teachers which specialize on individual tasks. In this setting kickstarting yields surprisingly large gains, with the kickstarted agent matching the performance of an agent trained from scratch in almost 10x fewer steps, and surpassing its final performance by 42 percent. Kickstarting is conceptually simple and can easily be incorporated into reinforcement learning experiments.

Key facts. Cited by 81 papers in the map (data through 14 Oct 2026). First posted on arXiv 10 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2019: 1; 2020: 0; 2021: 9; 2022: 14; 2023: 15; 2024: 20; 2025: 10; 2026: 12.

81 papers.

Sort:

By area:

  1. 2022Red Teaming Language Models with Language Modelscited by 1,026
  2. 2022Teaching language models to support answers with verified quotescited by 292
  3. 2021Open-Ended Learning Leads to Generally Capable Agentscited by 182
  4. 2023Understanding plasticity in neural networkscited by 174
  5. 2023Human-Timescale Adaptation in an Open-Ended Task Spacecited by 138
  6. 2023Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learningcited by 133
  7. 2023Universal Humanoid Motion Representations for Physics-Based Controlcited by 116
  8. 2022Automated Reinforcement Learning (AutoRL): A Survey and Open Problemscited by 70
  9. 2022Reincarnating Reinforcement Learning: Reusing Prior Computation to Accelerate Progresscited by 65
  10. 2025BREAD: Branched Rollouts from Expert Anchors Bridge SFT & RL for Reasoningcited by 63
  11. 2025Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Searchcited by 53
  12. 2024In value-based deep reinforcement learning, a pruned network is a good networkcited by 39
38 more
  1. 2021Alchemy: A benchmark and analysis toolkit for meta-reinforcement learning agentscited by 36
  2. 2021Lifelong Robotic Reinforcement Learning by Retaining Experiencescited by 34
  3. 2022RLx2: Training a Sparse Deep Reinforcement Learning Model from Scratchcited by 30
  4. 2023Large Language Model as a Policy Teacher for Training Reinforcement Learning Agentscited by 30
  5. 2023Evolutionary Reinforcement Learning: A Surveycited by 29
  6. 2021On Lottery Tickets and Minimal Task Representations in Deep Reinforcement Learningcited by 24
  7. 2024Fine-tuning Reinforcement Learning Models is Secretly a Forgetting Mitigation Problemcited by 23
  8. 2022Pretraining in Deep Reinforcement Learning: A Surveycited by 18
  9. 2023TGRL: An Algorithm for Teacher Guided Reinforcement Learningcited by 17
  10. 2024Real-Time Simulated Avatar from Head-Mounted Sensorscited by 17
  11. 2024Learning Robot Soccer from Egocentric Vision with Deep Reinforcement Learningcited by 17
  12. 2024When LLMs Play the Telephone Game: Cultural Attractors as Conceptual Tools to Evaluate LLMs in Multi-turn Settingscited by 17
  13. 2022Dungeons and Data: A Large-Scale NetHack Datasetcited by 16
  14. 2023Knowledge Distillation Performs Partial Variance Reductioncited by 11
  15. 2023Guarded Policy Optimization with Imperfect Online Demonstrationscited by 10
  16. 2022How to Spend Your Robot Time: Bridging Kickstarting and Offline Reinforcement Learning for Vision-based Robotic Manipulationcited by 9
  17. 2024Proximal Policy Distillationcited by 9
  18. 2024Bridging SFT and DPO for Diffusion Model Alignment with Self-Sampling Preference Optimizationcited by 9
  19. 2024Embodied CoT Distillation From LLM To Off-the-shelf Agentscited by 9
  20. 2025ThinkTuning: Instilling Cognitive Reflections without Distillationcited by 9
  21. 2021Tuning Mixed Input Hyperparameters on the Fly for Efficient Population Based AutoRLcited by 8
  22. 2024Artificial Generational Intelligence: Cultural Accumulation in Reinforcement Learningcited by 8
  23. 2024Reinforcement Learning Gradients as Vitamin for Online Finetuning Decision Transformerscited by 8
  24. 2022Hierarchical Kickstarting for Skill Transfer in Reinforcement Learningcited by 7
  25. 2023Visual-Policy Learning through Multi-Camera View to Single-Camera View Knowledge Distillation for Robot Manipulation Taskscited by 7
  26. 2022SkillS: Adaptive Skill Sequencing for Efficient Temporally-Extended Explorationcited by 6
  27. 2024All You Need in Knowledge Distillation Is a Tailored Coordinate Systemcited by 6
  28. 2019ZPD Teaching Strategies for Deep Reinforcement Learning from Demonstrationscited by 5
  29. 2022Data augmentation for efficient learning from parametric expertscited by 5
  30. 2021On Multi-objective Policy Optimization as a Tool for Reinforcement Learning: Case Studies in Offline RL and Finetuningcited by 4
  31. 2022Probing Transfer in Deep Reinforcement Learning without Task Engineeringcited by 4
  32. 2024Enhancing Reinforcement Learning Agents with Local Guidescited by 4
  33. 2024Generalized Population-Based Training for Hyperparameter Optimization in Reinforcement Learningcited by 4
  34. 2023Developing Driving Strategies Efficiently: A Skill-Based Hierarchical Reinforcement Learning Approachcited by 3
  35. 2024Combining RL and IL using a dynamic, performance-based modulation over learning signals and its application to local planningcited by 3
  36. 2025Sample Efficient Reinforcement Learning via Large Vision Language Model Distillationcited by 3
  37. 2026TerraTransfer: Learning End-to-End Driving Policies Without Expert Demonstrationscited by 3
  38. 2023Accelerating Policy Gradient by Estimating Value Function from Prior Computation in Deep Reinforcement Learningcited by 2

50 most cited of 81 All 81 (JSON)