# adaPARL: Adaptive Privacy-Aware Reinforcement Learning for Sequential-Decision Making Human-in-the-Loop Systems

arXiv: https://arxiv.org/abs/2303.04257 · HTML: https://mlsanity.com/paper/2303.04257/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 58 references that are in the map.
- First posted on arXiv 7 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [Cooperative Inverse Reinforcement Learning](https://mlsanity.com/paper/1606.03137/) (2016)
- [Privacy-preserving Q-Learning with Functional Noise in Continuous State Spaces](https://mlsanity.com/paper/1901.10634/) (2019)
- [How You Act Tells a Lot: Privacy-Leakage Attack on Deep Reinforcement Learning](https://mlsanity.com/paper/1904.11082/) (2019)
- [Local Differential Privacy for Regret Minimization in Reinforcement Learning](https://mlsanity.com/paper/2010.07778/) (2020)
- [Privacy Amplification via Shuffling for Linear Contextual Bandits](https://mlsanity.com/paper/2112.06008/) (2021)
- [FaiR-IoT: Fairness-aware Human-in-the-Loop Reinforcement Learning for Harnessing Human Variability in Personalized IoT](https://mlsanity.com/paper/2103.16033/) (2021)
- [Deceptive Reinforcement Learning for Privacy-Preserving Planning](https://mlsanity.com/paper/2102.03022/) (2021)
- [Privacy-Aware Time-Series Data Sharing with Deep Reinforcement Learning](https://mlsanity.com/paper/2003.02685/) (2020)
- [Two steps to risk sensitivity](https://mlsanity.com/paper/2111.06803/) (2021)
- [Privacy-Preserving Bandits](https://mlsanity.com/paper/1909.04421/) (2019)
- [Privacy-Constrained Policies via Mutual Information Regularized Policy Gradients](https://mlsanity.com/paper/2012.15019/) (2020)
- [VindiCo: Privacy Safeguard Against Adaptation Based Spyware in Human-in-the-Loop IoT](https://mlsanity.com/paper/2202.01348/) (2022)

## Built on by (most cited first, 6 of 6)

- [Privacy Risks in Reinforcement Learning for Household Robots](https://mlsanity.com/paper/2306.09273/) (2023)
- [FinP: Fairness-in-Privacy in Federated Learning by Addressing Disparities in Privacy Risk](https://mlsanity.com/paper/2502.17748/) (2025)
- [FAIRO: Fairness-aware Adaptation in Sequential-Decision Making for Human-in-the-Loop Systems](https://mlsanity.com/paper/2307.05857/) (2023)
- [PEARL: Structural Privacy-Utility Control in Human-Centric CPS via Personalized Early-Exit Deep Reinforcement Learning](https://mlsanity.com/paper/2403.05864/) (2024)
- [Position: Embodied AI Requires a Privacy-Utility Trade-off](https://mlsanity.com/paper/2605.05017/) (2026)
- [FinA: Fairness of Adverse Effects in Decision-Making of Human-Cyber-Physical-System](https://mlsanity.com/paper/2311.03468/) (2023)

## Abstract (arXiv)

Reinforcement learning (RL) presents numerous benefits compared to rule-based approaches in various applications. Privacy concerns have grown with the widespread use of RL trained with privacy-sensitive data in IoT devices, especially for human-in-the-loop systems. On the one hand, RL methods enhance the user experience by trying to adapt to the highly dynamic nature of humans. On the other hand, trained policies can leak the user's private information. Recent attention has been drawn to designing privacy-aware RL algorithms while maintaining an acceptable system utility. A central challenge in designing privacy-aware RL, especially for human-in-the-loop systems, is that humans have intrinsic variability and their preferences and behavior evolve. The effect of one privacy leak mitigation can be different for the same human or across different humans over time. Hence, we can not design one fixed model for privacy-aware RL that fits all. To that end, we propose adaPARL, an adaptive approach for privacy-aware RL, especially for human-in-the-loop IoT systems. adaPARL provides a personalized privacy-utility trade-off depending on human behavior and preference. We validate the proposed adaPARL on two IoT applications, namely (i) Human-in-the-Loop Smart Home and (ii) Human-in-the-Loop Virtual Reality (VR) Smart Classroom. Results obtained on these two applications validate the generality of adaPARL and its ability to provide a personalized privacy-utility trade-off. On average, for the first application, adaPARL improves the utility by 57% over the baseline and by 43% over randomization. adaPARL also reduces the privacy leak by 23% on average. For the second application, adaPARL decreases the privacy leak to 44% before the utility drops by 15%.