# Reinforcement Learning for Adaptive Caching with Dynamic Storage Pricing

arXiv: https://arxiv.org/abs/1812.08593 · HTML: https://mlsanity.com/paper/1812.08593/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2018.

## Built on by (most cited first, 8 of 8)

- [Optimistic No-regret Algorithms for Discrete Caching](https://mlsanity.com/paper/2208.06414/) (2022)
- [Online Caching with Optimistic Learning](https://mlsanity.com/paper/2202.10590/) (2022)
- [Cocktail Edge Caching: Ride Dynamic Trends of Content Popularity with Ensemble Learning](https://mlsanity.com/paper/2101.05885/) (2021)
- [Online Caching with no Regret: Optimistic Learning via Recommendations](https://mlsanity.com/paper/2204.09345/) (2022)
- [Predictive Edge Caching through Deep Mining of Sequential Patterns in User Content Retrievals](https://mlsanity.com/paper/2210.02657/) (2022)
- [A Survey on Reinforcement Learning-Aided Caching in Mobile Edge Networks](https://mlsanity.com/paper/2105.05564/) (2021)
- [Robust, Deep, and Reinforcement Learning for Management of Communication and Power Networks](https://mlsanity.com/paper/2202.05395/) (2022)
- [From Traditional Adaptive Data Caching to Adaptive Context Caching: A Survey](https://mlsanity.com/paper/2211.11259/) (2022)

## Abstract (arXiv)

Small base stations (SBs) of fifth-generation (5G) cellular networks are envisioned to have storage devices to locally serve requests for reusable and popular contents by caching them at the edge of the network, close to the end users. The ultimate goal is to shift part of the predictable load on the back-haul links, from on-peak to off-peak periods, contributing to a better overall network performance and service experience. To enable the SBs with efficient textitfetch-cache decision-making schemes operating in dynamic settings, this paper introduces simple but flexible generic time-varying fetching and caching costs, which are then used to formulate a constrained minimization of the aggregate cost across files and time. Since caching decisions per time slot influence the content availability in future slots, the novel formulation for optimal fetch-cache decisions falls into the class of dynamic programming. Under this generic formulation, first by considering stationary distributions for the costs and file popularities, an efficient reinforcement learning-based solver known as value iteration algorithm can be used to solve the emerging optimization problem. Later, it is shown that practical limitations on cache capacity can be handled using a particular instance of the generic dynamic pricing formulation. Under this setting, to provide a light-weight online solver for the corresponding optimization, the well-known reinforcement learning algorithm, Q-learning, is employed to find optimal fetch-cache decisions. Numerical tests corroborating the merits of the proposed approach wrap up the paper.