# BRNES: Enabling Security and Privacy-aware Experience Sharing in Multiagent Robotic and Autonomous Systems

arXiv: https://arxiv.org/abs/2308.01274 · HTML: https://mlsanity.com/paper/2308.01274/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 2 Aug 2023.

## Builds on (most cited first, 13 of 13)

- [Deep Reinforcement Learning for Autonomous Driving: A Survey](https://mlsanity.com/paper/2002.00444/) (2020)
- [QMIX: Monotonic Value Function Factorisation for Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1803.11485/) (2018)
- [Adaptive Reward-Poisoning Attacks against Reinforcement Learning](https://mlsanity.com/paper/2003.12613/) (2020)
- [Coordinated Multi-Agent Imitation Learning](https://mlsanity.com/paper/1703.03121/) (2017)
- [Privacy-preserving Q-Learning with Functional Noise in Continuous State Spaces](https://mlsanity.com/paper/1901.10634/) (2019)
- [Multi-agent Reinforcement Learning for Cooperative Lane Changing of Connected and Autonomous Vehicles in Mixed Traffic](https://mlsanity.com/paper/2111.06318/) (2021)
- [Adversarial attacks in consensus-based multi-agent reinforcement learning](https://mlsanity.com/paper/2103.06967/) (2021)
- [A Q-values Sharing Framework for Multiagent Reinforcement Learning under Budget Constraint](https://mlsanity.com/paper/2011.14281/) (2020)
- [Differential Advising in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2011.03640/) (2020)
- [How Private Is Your RL Policy? An Inverse RL Based Analysis Framework](https://mlsanity.com/paper/2112.05495/) (2021)
- [Multi-agent Deep Reinforcement Learning for Zero Energy Communities](https://mlsanity.com/paper/1810.03679/) (2018)
- [Privacy, Security, and Utility Analysis of Differentially Private CPES Data](https://mlsanity.com/paper/2109.09963/) (2021)
- [Improved Reinforcement Learning in Cooperative Multi-agent Environments Using Knowledge Transfer](https://mlsanity.com/paper/2107.09807/) (2021)

## Built on by (most cited first, 1 of 1)

- [Differentially Private Reinforcement Learning with Self-Play](https://mlsanity.com/paper/2404.07559/) (2024)

## Abstract (arXiv)

Although experience sharing (ES) accelerates multiagent reinforcement learning (MARL) in an advisor-advisee framework, attempts to apply ES to decentralized multiagent systems have so far relied on trusted environments and overlooked the possibility of adversarial manipulation and inference. Nevertheless, in a real-world setting, some Byzantine attackers, disguised as advisors, may provide false advice to the advisee and catastrophically degrade the overall learning performance. Also, an inference attacker, disguised as an advisee, may conduct several queries to infer the advisors' private information and make the entire ES process questionable in terms of privacy leakage. To address and tackle these issues, we propose a novel MARL framework (BRNES) that heuristically selects a dynamic neighbor zone for each advisee at each learning step and adopts a weighted experience aggregation technique to reduce Byzantine attack impact. Furthermore, to keep the agent's private information safe from adversarial inference attacks, we leverage the local differential privacy (LDP)-induced noise during the ES process. Our experiments show that our framework outperforms the state-of-the-art in terms of the steps to goal, obtained reward, and time to goal metrics. Particularly, our evaluation shows that the proposed framework is 8.32x faster than the current non-private frameworks and 1.41x faster than the private frameworks in an adversarial setting.