mlsanityData to 14 Oct 2026

Privacy-Preserving Kickstarting Deep Reinforcement Learning with Privacy-Aware Learners

arXiv 2102.09599 ↗First posted 18 Feb 2021cs.LG cs.AI cs.CRcited by 1

On the map: Knowledge distillation → Federated, robust and private distillation

Abstract (arXiv)

Kickstarting deep reinforcement learning algorithms facilitate a teacher-student relationship among the agents and allow for a well-performing teacher to share demonstrations with a student to expedite the student's training. However, despite the known benefits, the demonstrations may contain sensitive information about the teacher's training data and existing kickstarting methods do not take any measures to protect it. Therefore, we use the framework of differential privacy to develop a mechanism that securely shares the teacher's demonstrations with the student. The mechanism allows for the teacher to decide upon the accuracy of its demonstrations with respect to the privacy budget that it consumes, thereby granting the teacher full control over its data privacy. We then develop a kickstarted deep reinforcement learning algorithm for the student that is privacy-aware because we calibrate its objective with the parameters of the teacher's privacy mechanism. The privacy-aware design of the algorithm makes it possible to kickstart the student's learning despite the perturbations induced by the privacy mechanism. From numerical experiments, we highlight three empirical results: (i) the algorithm succeeds in expediting the student's learning, (ii) the student converges to a performance level that was not possible without the demonstrations, and (iii) the student maintains its enhanced performance even after the teacher stops sharing useful demonstrations due to its privacy budget constraints.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in Federated, robust and private distillation, one of 12 directions in Knowledge distillation. Builds on 14 of its 26 references that are in the map. First posted on arXiv 18 Feb 2021.

Builds on

14 of 26 references.

References · 14

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2013Intriguing properties of neural networkscited by 5,155
  3. 2016Deep Learning with Differential Privacycited by 2,728
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  6. 2016OpenAI Gymcited by 1,736
  7. 2015Policy Distillationcited by 440
  8. 2016Semi-supervised Knowledge Transfer for Deep Learning from Private Training Datacited by 348
  9. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 293
  10. 2015Actor-Mimic: Deep Multitask and Transfer Reinforcement Learningcited by 178
  11. 2018Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendationcited by 101
  12. 2018Kickstarting Deep Reinforcement Learningcited by 81
2 more
  1. 2019How You Act Tells a Lot: Privacy-Leakage Attack on Deep Reinforcement Learningcited by 29
  2. 2020Privacy-Preserving Policy Synthesis in Markov Decision Processescited by 7

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Private Knowledge Sharing in Distributed Learning: A Surveycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025On the Sample Complexity of Differentially Private Policy Optimization
  2. 2026Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modeling
  3. 2025Differentially Private Policy Gradient
  4. 2024Preserving Expert-Level Privacy in Offline Reinforcement Learning
  5. 2024Privacy-Preserving Student Learning with Differentially Private Data-Free Distillation
  6. 2026Privacy Preserving Reinforcement Learning with One-Sided Feedback
  7. 2024Near-Optimal Reinforcement Learning with Shuffle Differential Privacy
  8. 2026Reward-Driven Learning under Prompt-Level Differential Privacy