# Edge Caching Optimization with PPO and Transfer Learning for Dynamic Environments

arXiv: https://arxiv.org/abs/2411.09812 · HTML: https://mlsanity.com/paper/2411.09812/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 14 Nov 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 9 of 9)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Deep Learning for Wireless Coded Caching with Unknown and Time-Variant Content Popularity](https://mlsanity.com/paper/2008.09422/) (2020)
- [Cocktail Edge Caching: Ride Dynamic Trends of Content Popularity with Ensemble Learning](https://mlsanity.com/paper/2101.05885/) (2021)
- [The Design of Dynamic Probabilistic Caching with Time-Varying Content Popularity](https://mlsanity.com/paper/2002.06251/) (2020)
- [Learning from Peers: Deep Transfer Reinforcement Learning for Joint Radio and Cache Resource Allocation in 5G RAN Slicing](https://mlsanity.com/paper/2109.07999/) (2021)
- [Reinforcement Learning Based Cooperative Coded Caching under Dynamic Popularities in Ultra-Dense Networks](https://mlsanity.com/paper/2003.03758/) (2020)
- [A Deep Reinforcement Learning-Based Caching Strategy for IoT Networks with Transient Data](https://mlsanity.com/paper/2203.12674/) (2022)
- [An SMDP-Based Approach to Thermal-Aware Task Scheduling in NoC-based MPSoC platforms](https://mlsanity.com/paper/2009.02813/) (2020)
- [Attention-Enhanced Prioritized Proximal Policy Optimization for Adaptive Edge Caching](https://mlsanity.com/paper/2402.14576/) (2024)

## Built on by (most cited first, 1 of 1)

- [Fast-Convergent Meta-RL via Gradient-Clustered BS Sampling for Edge Caching](https://mlsanity.com/paper/2609.16370/) (2026)

## Abstract (arXiv)

This paper addresses the challenge of edge caching in dynamic environments, where rising traffic loads strain backhaul links and core networks. We propose a Proximal Policy Optimization (PPO)-based caching strategy that fully incorporates key file attributes such as size, lifetime, importance, and popularity, while also considering random file request arrivals, reflecting more realistic edge caching scenarios. In dynamic environments, changes such as shifts in content popularity and variations in request rates frequently occur, making previously learned policies less effective as they were optimized for earlier conditions. Without adaptation, caching efficiency and response times can degrade. While learning a new policy from scratch in a new environment is an option, it is highly inefficient and computationally expensive. Thus, adapting an existing policy to these changes is critical. To address this, we develop a mechanism that detects changes in content popularity and request rates, ensuring timely adjustments to the caching strategy. We also propose a transfer learning-based PPO algorithm that accelerates convergence in new environments by leveraging prior knowledge. Simulation results demonstrate the significant effectiveness of our approach, outperforming a recent Deep Reinforcement Learning (DRL)-based method.