Differentially Private Reinforcement Learning with Self-Play
Not on the map. Why?
Abstract (arXiv)
We study the problem of multi-agent reinforcement learning (multi-agent RL) with differential privacy (DP) constraints. This is well-motivated by various real-world applications involving sensitive data, where it is critical to protect users' private information. We first extend the definitions of Joint DP (JDP) and Local DP (LDP) to two-player zero-sum episodic Markov Games, where both definitions ensure trajectory-wise privacy protection. Then we design a provably efficient algorithm based on optimistic Nash value iteration and privatization of Bernstein-type bonuses. The algorithm is able to satisfy JDP and LDP requirements when instantiated with appropriate privacy mechanisms. Furthermore, for both notions of DP, our regret bound generalizes the best known result under the single-agent RL case, while our regret could also reduce to the best known result for multi-agent RL without privacy constraints. To the best of our knowledge, these are the first line of results towards understanding trajectory-wise privacy protection in multi-agent RL.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 52 of its 65 references that are in the map. First posted on arXiv 11 Apr 2024.
Builds on
52 of 65 references.
References · 52
- 2018The Secret Sharer: Evaluating and Testing Unintended Memorization in Neural Networkscited by 847
- 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 514
- 2017Minimax Regret Bounds for Reinforcement Learningcited by 500
- 2018Is Q-learning Provably Efficient?cited by 485
- 2016Concentrated Differential Privacy: Simplifications, Extensions, and Lower Boundscited by 339
- 2016Safe, Multi-Agent, Reinforcement Learning for Autonomous Drivingcited by 301
- 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 233
- 2017Unifying PAC and Regret: Uniform PAC Bounds for Episodic Reinforcement Learningcited by 175
- 2021Practical and Private (Deep) Learning without Sampling or Shufflingcited by 169
- 2020Provable Self-Play Algorithms for Competitive Reinforcement Learningcited by 112
- 2021V-Learning -- A Simple, Efficient, Decentralized Algorithm for Multiagent RLcited by 111
- 2020Near-Optimal Reinforcement Learning with Self-Playcited by 108
38 more
- 2020A Sharp Analysis of Model-based Reinforcement Learning with Self-Playcited by 104
- 2020When is Memorization of Irrelevant Training Data Necessary for High-Accuracy Learning?cited by 102
- 2020Learning Zero-Sum Simultaneous-Move Markov Games Using Function Approximation and Correlated Equilibriumcited by 100
- 2018Differentially Private Contextual Linear Banditscited by 82
- 2020Towards Playing Full MOBA Games with Deep Reinforcement Learningcited by 78
- 2020Private Reinforcement Learning with PAC and Regret Guaranteescited by 51
- 2021On Improving Model-Free Algorithms for Decentralized Multi-Agent Reinforcement Learningcited by 35
- 2022Sample-Efficient Reinforcement Learning with loglog(T) Switching Costcited by 32
- 2019Privacy-preserving Q-Learning with Functional Noise in Continuous State Spacescited by 29
- 2023Breaking the Curse of Multiagency: Provably Efficient Decentralized Multi-Agent RL with Function Approximationcited by 29
- 2023Breaking the Curse of Multiagents in a Large State Space: RL in Markov Games with Independent Linear Function Approximationcited by 26
- 2013Private Matchings and Allocationscited by 23
- 2021Differentially Private Regret Minimization in Episodic Markov Decision Processescited by 23
- 2020Local Differential Privacy for Regret Minimization in Reinforcement Learningcited by 22
- 2022Differentially Private Reinforcement Learning with Linear Function Approximationcited by 22
- 2016Differentially Private Policy Evaluationcited by 19
- 2022Offline Reinforcement Learning with Differential Privacycited by 19
- 2023Privately Aligning Language Models with Reinforcement Learningcited by 17
- 2022Near-Optimal Differentially Private Reinforcement Learningcited by 16
- 2020Negotiating Team Formation Using Deep Reinforcement Learningcited by 13
- 2021Locally Differentially Private Reinforcement Learning for Linear Mixture Markov Decision Processescited by 13
- 2022Near-Optimal Deployment Efficiency in Reward-Free Reinforcement Learning with Linear Function Approximationcited by 13
- 2023Differentially Private Reward Estimation with Preference Feedbackcited by 12
- 2021Differentially Private Exploration in Reinforcement Learning with Linear Representationcited by 11
- 2022Differentially Private Linear Sketches: Efficient Implementations and Applicationscited by 10
- 2020Locally Private Distributed Reinforcement Learningcited by 9
- 2022Improved Regret for Differentially Private Exploration in Linear MDPcited by 8
- 2023Logarithmic Switching Cost in Reinforcement Learning beyond Linear MDPscited by 8
- 2021Adaptive Control of Differentially Private Linear Quadratic Systemscited by 7
- 2022Differentially Private Temporal Difference Learning with Stochastic Nonconvex-Strongly-Concave Optimizationcited by 7
- 2023Privacy Preserving Large Language Models: ChatGPT Case Study Based Vision and Frameworkcited by 7
- 2020Privacy-Constrained Policies via Mutual Information Regularized Policy Gradientscited by 6
- 2019Actor Critic with Differentially Private Criticcited by 5
- 2023Differentially Private Episodic Reinforcement Learning with Heavy-tailed Rewardscited by 4
- 2019Privacy Preserving Off-Policy Evaluationcited by 3
- 2024Near-Optimal Reinforcement Learning with Self-Play under Adaptivity Constraintscited by 3
- 2023Hiding in Plain Sight: Differential Privacy Noise Exploitation for Evasion-resilient Localized Poisoning Attacks in Multiagent Reinforcement Learningcited by 2
- 2023BRNES: Enabling Security and Privacy-aware Experience Sharing in Multiagent Robotic and Autonomous Systemscited by 1
50 most cited of 52 All 52 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.