mlsanityData to 14 Oct 2026

Locally Differentially Private Reinforcement Learning for Linear Mixture Markov Decision Processes

arXiv 2110.10133 ↗First posted 19 Oct 2021cs.LG cs.CR math.OCcited by 13

Not on the map. Why?

Abstract (arXiv)

Reinforcement learning (RL) algorithms can be used to provide personalized services, which rely on users' private and sensitive data. To protect the users' privacy, privacy-preserving RL algorithms are in demand. In this paper, we study RL with linear function approximation and local differential privacy (LDP) guarantees. We propose a novel (varepsilon, δ)-LDP algorithm for learning a class of Markov decision processes (MDPs) dubbed linear mixture MDPs, and obtains an tildeO( d5/4H7/4T3/4left(log(1/δ)right)1/4√1/varepsilon) regret, where d is the dimension of feature mapping, H is the length of the planning horizon, and T is the number of interactions with the environment. We also prove a lower bound Omega(dH√T/left(evarepsilon(evarepsilon-1)right)) for learning linear mixture MDPs under varepsilon-LDP constraint. Experiments on synthetic datasets verify the effectiveness of our algorithm. To the best of our knowledge, this is the first provable privacy-preserving RL algorithm with linear function approximation.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 19 Oct 2021.

Builds on

24 of 51 references.

References · 24

  1. 2016Deep Learning with Differential Privacycited by 2,728
  2. 2016Membership Inference Attacks against Machine Learning Modelscited by 2,281
  3. 2019Provably Efficient Reinforcement Learning with Linear Function Approximationcited by 520
  4. 2017Minimax Regret Bounds for Reinforcement Learningcited by 503
  5. 2016Contextual Decision Processes with Low Bellman Rank are PAC-Learnablecited by 292
  6. 2020Model-Based Reinforcement Learning with Value-Targeted Regressioncited by 234
  7. 2019Provably Efficient Exploration in Policy Optimizationcited by 205
  8. 2019Reinforcement Learning in Feature Space: Matrix Bandit, Kernels, and Regret Boundcited by 201
  9. 2020Nearly Minimax Optimal Reinforcement Learning for Linear Mixture Markov Decision Processescited by 200
  10. 2020Learning Near Optimal Policies with Low Inherent Bellman Errorcited by 181
  11. 2019Optimism in Reinforcement Learning with Generalized Linear Function Approximationcited by 111
  12. 2016Minimax Optimal Procedures for Locally Private Estimationcited by 107
12 more
  1. 2019A Short Note on Concentration Inequalities for Random Vectors with SubGaussian Normcited by 106
  2. 2020Provably Efficient Reinforcement Learning for Discounted MDPs with Feature Mappingcited by 102
  3. 2019Sample Complexity of Reinforcement Learning using Linearly Combined Model Ensemblescited by 100
  4. 2018Differentially Private Contextual Linear Banditscited by 82
  5. 2020Logarithmic Regret for Reinforcement Learning with Linear Function Approximationcited by 79
  6. 2020Locally Differentially Private (Contextual) Bandits Learningcited by 53
  7. 2020Private Reinforcement Learning with PAC and Regret Guaranteescited by 51
  8. 2019Differential Privacy for Multi-armed Bandits: What Is It and What Is Its Cost?cited by 27
  9. 2013Local Privacy, Data Processing Inequalities, and Statistical Minimax Ratescited by 26
  10. 2016Differentially Private Policy Evaluationcited by 19
  11. 2021Nearly Minimax Optimal Regret for Learning Infinite-horizon Average-reward MDPs with Linear Function Approximationcited by 18
  12. 2020(Locally) Differentially Private Combinatorial Semi-Banditscited by 14

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 5; 2023: 2; 2024: 5.

13 papers.

Sort:

By area:

  1. 2022Shuffle Private Linear Contextual Banditscited by 27
  2. 2022Differentially Private Reinforcement Learning with Linear Function Approximationcited by 22
  3. 2022Offline Reinforcement Learning with Differential Privacycited by 20
  4. 2022Near-Optimal Differentially Private Reinforcement Learningcited by 16
  5. 2022Distributed Differential Privacy in Multi-Armed Banditscited by 14
  6. 2021Differentially Private Exploration in Reinforcement Learning with Linear Representationcited by 11
  7. 2023Differentially Private Episodic Reinforcement Learning with Heavy-tailed Rewardscited by 4
  8. 2023DPMAC: Differentially Private Communication for Cooperative Multi-Agent Reinforcement Learningcited by 1
  9. 2024Differentially Private Deep Model-Based Reinforcement Learningcited by 1
  10. 2024Differentially Private Reinforcement Learning with Self-Playcited by 1
  11. 2024Privacy Preserving Reinforcement Learning for Population Processescited by 1
  12. 2024Near-Optimal Reinforcement Learning with Shuffle Differential Privacycited by 1
1 more
  1. 2024Preserving Expert-Level Privacy in Offline Reinforcement Learningcited by 0