# Attention-Enhanced Prioritized Proximal Policy Optimization for Adaptive Edge Caching

arXiv: https://arxiv.org/abs/2402.14576 · HTML: https://mlsanity.com/paper/2402.14576/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods.
- Builds on 4 of its 36 references that are in the map.
- First posted on arXiv 8 Feb 2024.

## Placed in

- Deep reinforcement learning methods / [RL for energy, networks and operations](https://mlsanity.com/deeprl/rl-for-energy-networks-and-operations/)

## Builds on (most cited first, 4 of 4)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [The Design of Dynamic Probabilistic Caching with Time-Varying Content Popularity](https://mlsanity.com/paper/2002.06251/) (2020)
- [A Deep Reinforcement Learning-Based Caching Strategy for IoT Networks with Transient Data](https://mlsanity.com/paper/2203.12674/) (2022)
- [An SMDP-Based Approach to Thermal-Aware Task Scheduling in NoC-based MPSoC platforms](https://mlsanity.com/paper/2009.02813/) (2020)

## Built on by (most cited first, 2 of 2)

- [Edge Caching Optimization with PPO and Transfer Learning for Dynamic Environments](https://mlsanity.com/paper/2411.09812/) (2024)
- [Fast-Convergent Meta-RL via Gradient-Clustered BS Sampling for Edge Caching](https://mlsanity.com/paper/2609.16370/) (2026)

## Abstract (arXiv)

This paper tackles the growing issue of excessive data transmission in networks. With increasing traffic, backhaul links and core networks are under significant traffic, leading to the investigation of caching solutions at edge routers. Many existing studies utilize Markov Decision Processes (MDP) to tackle caching problems, often assuming decision points at fixed intervals; however, real-world environments are characterized by random request arrivals. Additionally, critical file attributes such as lifetime, size, and priority significantly impact the effectiveness of caching policies, yet existing research fails to integrate all these attributes in policy design. In this work, we model the caching problem using a Semi-Markov Decision Process (SMDP) to better capture the continuous-time nature of real-world applications, enabling caching decisions to be triggered by random file requests. We then introduce a Proximal Policy Optimization (PPO)--based caching strategy that fully considers file attributes like lifetime, size, and priority. Simulations show that our method outperforms a recent Deep Reinforcement Learning-based technique. To further advance our research, we improved the convergence rate of PPO by prioritizing transitions within the replay buffer through an attention mechanism. This mechanism evaluates the similarity between the current state and all stored transitions, assigning higher priorities to transitions that exhibit greater similarity.