# Federated Reinforcement Distillation with Proxy Experience Memory

arXiv: https://arxiv.org/abs/1907.06536 · HTML: https://mlsanity.com/paper/1907.06536/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 5 of 5)

- [Federated Distillation: A Survey](https://mlsanity.com/paper/2404.08564/) (2024)
- [Autonomy and Intelligence in the Computing Continuum: Challenges, Enablers, and Future Directions for Orchestration](https://mlsanity.com/paper/2205.01423/) (2022)
- [Towards Open Federated Learning Platforms: Survey and Vision from Technical and Legal Perspectives](https://mlsanity.com/paper/2307.02140/) (2023)
- [Momentum-Based Federated Reinforcement Learning with Interaction and Communication Efficiency](https://mlsanity.com/paper/2405.17471/) (2024)
- [Federated Offline Policy Optimization with Dual Regularization](https://mlsanity.com/paper/2405.17474/) (2024)

## Abstract (arXiv)

In distributed reinforcement learning, it is common to exchange the experience memory of each agent and thereby collectively train their local models. The experience memory, however, contains all the preceding state observations and their corresponding policies of the host agent, which may violate the privacy of the agent. To avoid this problem, in this work, we propose a privacy-preserving distributed reinforcement learning (RL) framework, termed federated reinforcement distillation (FRD). The key idea is to exchange a proxy experience memory comprising a pre-arranged set of states and time-averaged policies, thereby preserving the privacy of actual experiences. Based on an advantage actor-critic RL architecture, we numerically evaluate the effectiveness of FRD and investigate how the performance of FRD is affected by the proxy memory structure and different memory exchanging rules.