# Distributed Reinforcement Learning for Privacy-Preserving Dynamic Edge Caching

arXiv: https://arxiv.org/abs/2110.10349 · HTML: https://mlsanity.com/paper/2110.10349/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 30 references that are in the map.
- First posted on arXiv 20 Oct 2021.

## Builds on (most cited first, 12 of 12)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Advances and Open Problems in Federated Learning](https://mlsanity.com/paper/1912.04977/) (2019)
- [Continuous Deep Q-Learning with Model-based Acceleration](https://mlsanity.com/paper/1603.00748/) (2016)
- [Optimal and Scalable Caching for 5G Using Reinforcement Learning of Space-time Popularities](https://mlsanity.com/paper/1708.06698/) (2017)
- [Edge Computing For Smart Health: Context-aware Approaches, Opportunities, and Challenges](https://mlsanity.com/paper/2004.07311/) (2020)
- [Security in Mobile Edge Caching with Reinforcement Learning](https://mlsanity.com/paper/1801.05915/) (2018)
- [Cellular-Connected Wireless Virtual Reality: Requirements, Challenges, and Solutions](https://mlsanity.com/paper/2001.06287/) (2020)
- [Deep Reinforcement Learning and Permissioned Blockchain for Content Caching in Vehicular Edge Computing and Networks](https://mlsanity.com/paper/2011.08449/) (2020)
- [Hybrid Policy Learning for Energy-Latency Tradeoff in MEC-Assisted VR Video Service](https://mlsanity.com/paper/2104.01036/) (2021)
- [Understanding Performance of Edge Content Caching for Mobile Video Streaming](https://mlsanity.com/paper/1702.07627/) (2017)
- [Capacity-Aware Edge Caching in Fog Computing Networks](https://mlsanity.com/paper/2002.02173/) (2020)
- [Online Edge Caching and Wireless Delivery in Fog-Aided Networks with Dynamic Content Popularity](https://mlsanity.com/paper/1711.10430/) (2017)

## Built on by (most cited first, 10 of 10)

- [Trustworthy Federated Learning: Privacy, Security, and Beyond](https://mlsanity.com/paper/2411.01583/) (2024)
- [FedGiA: An Efficient Hybrid Algorithm for Federated Learning](https://mlsanity.com/paper/2205.01438/) (2022)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [New Challenges in Reinforcement Learning: A Survey of Security and Privacy](https://mlsanity.com/paper/2301.00188/) (2022)
- [Unsupervised Recurrent Federated Learning for Edge Popularity Prediction in Privacy-Preserving Mobile Edge Computing Networks](https://mlsanity.com/paper/2207.00755/) (2022)
- [Multi-Content Time-Series Popularity Prediction with Multiple-Model Transformers in MEC Networks](https://mlsanity.com/paper/2210.05874/) (2022)
- [Fractional Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing](https://mlsanity.com/paper/2312.10418/) (2023)
- [Intelligent Hybrid Resource Allocation in MEC-assisted RAN Slicing Network](https://mlsanity.com/paper/2405.17436/) (2024)
- [Asynchronous Fractional Multi-Agent Deep Reinforcement Learning for Age-Minimal Mobile Edge Computing](https://mlsanity.com/paper/2409.16832/) (2024)
- [Learning for Cross-Layer Resource Allocation in MEC-Aided Cell-Free Networks](https://mlsanity.com/paper/2412.16565/) (2024)

## Abstract (arXiv)

Mobile edge computing (MEC) is a prominent computing paradigm which expands the application fields of wireless communication. Due to the limitation of the capacities of user equipments and MEC servers, edge caching (EC) optimization is crucial to the effective utilization of the caching resources in MEC-enabled wireless networks. However, the dynamics and complexities of content popularities over space and time as well as the privacy preservation of users pose significant challenges to EC optimization. In this paper, a privacy-preserving distributed deep deterministic policy gradient (P2D3PG) algorithm is proposed to maximize the cache hit rates of devices in the MEC networks. Specifically, we consider the fact that content popularities are dynamic, complicated and unobservable, and formulate the maximization of cache hit rates on devices as distributed problems under the constraints of privacy preservation. In particular, we convert the distributed optimizations into distributed model-free Markov decision process problems and then introduce a privacy-preserving federated learning method for popularity prediction. Subsequently, a P2D3PG algorithm is developed based on distributed reinforcement learning to solve the distributed problems. Simulation results demonstrate the superiority of the proposed approach in improving EC hit rate over the baseline methods while preserving user privacy.