mlsanityData to 14 Oct 2026

Privacy Preserving Reinforcement Learning for Population Processes

arXiv 2406.17649 ↗First posted 25 Jun 2024cs.LG cs.CRcited by 1

On the map: Deep reinforcement learning methods → RL for energy, networks and operations

Abstract (arXiv)

We consider the problem of privacy protection in Reinforcement Learning (RL) algorithms that operate over population processes, a practical but understudied setting that includes, for example, the control of epidemics in large populations of dynamically interacting individuals. In this setting, the RL algorithm interacts with the population over T time steps by receiving population-level statistics as state and performing actions which can affect the entire population at each time step. An individual's data can be collected across multiple interactions and their privacy must be protected at all times. We clarify the Bayesian semantics of Differential Privacy (DP) in the presence of correlated data in population processes through a Pufferfish Privacy analysis. We then give a meta algorithm that can take any RL algorithm as input and make it differentially private. This is achieved by taking an approach that uses DP mechanisms to privatize the state and reward signal at each time step before the RL algorithm receives them as input. Our main theoretical result shows that the value-function approximation error when applying standard RL algorithms directly to the privatized states shrinks quickly as the population size and privacy budget increase. This highlights that reasonable privacy-utility trade-offs are possible for differentially private RL algorithms in population processes. Our theoretical findings are validated by experiments performed on a simulated epidemic control problem over large population sizes.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 34 of its 77 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

34 of 77 references.

References · 34

  1. 2020The Discrete Gaussian for Differential Privacycited by 135
  2. 2020Differentially-Private Federated Linear Banditscited by 83
  3. 2018Differentially Private Contextual Linear Banditscited by 82
  4. 2014Epidemic processes in complex networkscited by 58
  5. 2020Locally Differentially Private (Contextual) Bandits Learningcited by 53
  6. 2020Private Reinforcement Learning with PAC and Regret Guaranteescited by 51
  7. 2021Twitch Gamers: a Dataset for Evaluating Proximity Preserving and Structural Role-based Node Embeddingscited by 49
  8. 2021Partially Observable Markov Decision Processes (POMDPs) and Roboticscited by 49
  9. 2015Algorithms for Differentially Private Multi-Armed Banditscited by 45
  10. 2019An Optimal Private Stochastic-MAB Algorithm Based on an Optimal Private Stopping Rulecited by 37
  11. 2020Multi-Armed Bandits with Local Differential Privacycited by 37
  12. 2019Privacy-preserving Q-Learning with Functional Noise in Continuous State Spacescited by 30
22 more
  1. 2019How You Act Tells a Lot: Privacy-Leakage Attack on Deep Reinforcement Learningcited by 29
  2. 2022When Privacy Meets Partial Information: A Refined Analysis of Differentially Private Banditscited by 28
  3. 2020Reinforcement Learning for Optimization of COVID-19 Mitigation policiescited by 25
  4. 2017Achieving Privacy in the Adversarial Multi-Armed Banditcited by 24
  5. 2021Differentially Private Regret Minimization in Episodic Markov Decision Processescited by 23
  6. 2020Local Differential Privacy for Regret Minimization in Reinforcement Learningcited by 22
  7. 2022Differentially Private Reinforcement Learning with Linear Function Approximationcited by 22
  8. 2016Differentially Private Policy Evaluationcited by 19
  9. 2022Near-Optimal Differentially Private Reinforcement Learningcited by 16
  10. 2021Private Graph Data Release: A Surveycited by 15
  11. 2015Analysis and Control of Epidemics: A survey of spreading processes on complex networkscited by 14
  12. 2022Distributed Differential Privacy in Multi-Armed Banditscited by 14
  13. 2018The Bounded Laplace Mechanism in Differential Privacycited by 13
  14. 2021Locally Differentially Private Reinforcement Learning for Linear Mixture Markov Decision Processescited by 13
  15. 2020Real-world Video Adaptation with Reinforcement Learningcited by 12
  16. 2020EpidemiOptim: A Toolbox for the Optimization of Control Policies in Epidemiological Modelscited by 11
  17. 2021Differentially Private Exploration in Reinforcement Learning with Linear Representationcited by 11
  18. 2022Improved Regret for Differentially Private Exploration in Linear MDPcited by 8
  19. 2021Robo-Advising: Enhancing Investment with Inverse Optimization and Deep Reinforcement Learningcited by 7
  20. 2020COVID-19 Pandemic Cyclic Lockdown Optimization Using Reinforcement Learningcited by 6
  21. 2020COVID-19 pandemic control: balancing detection policy and lockdown intervention under ICU sustainabilitycited by 3
  22. 2022Belief Acquisition as Stochastic Filteringcited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Near-Optimal Reinforcement Learning with Shuffle Differential Privacycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Offline Reinforcement Learning with Differential Privacy
  2. 2026Privacy-Preserving Reinforcement Learning from Human Feedback via Decoupled Reward Modeling
  3. 2025On the Sample Complexity of Differentially Private Policy Optimization
  4. 2024Differentially Private Reinforcement Learning with Self-Play
  5. 2024Preserving Expert-Level Privacy in Offline Reinforcement Learning
  6. 2026Randomized Least Squares Value Iteration itself is Joint Differentially Private
  7. 2025Differentially Private Policy Gradient
  8. 2025Position Paper: Rethinking Privacy in RL for Sequential Decision-making in the Age of LLMs