# Markov Decision Policies for Dynamic Video Delivery in Wireless Caching Networks

arXiv: https://arxiv.org/abs/1902.10932 · HTML: https://mlsanity.com/paper/1902.10932/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2019.

## Built on by (most cited first, 8 of 8)

- [Quantum Neural Networks: Concepts, Applications, and Challenges](https://mlsanity.com/paper/2108.01468/) (2021)
- [Cooperative Multi-Agent Deep Reinforcement Learning for Reliable and Energy-Efficient Mobile Access via Multi-UAV Control](https://mlsanity.com/paper/2210.00945/) (2022)
- [Multi-Agent Deep Reinforcement Learning using Attentive Graph Neural Architectures for Real-Time Strategy Games](https://mlsanity.com/paper/2105.10211/) (2021)
- [Spatio-Temporal Split Learning for Autonomous Aerial Surveillance using Urban Air Mobility (UAM) Networks](https://mlsanity.com/paper/2111.11856/) (2021)
- [Parallelized and Randomized Adversarial Imitation Learning for Safety-Critical Self-Driving Vehicles](https://mlsanity.com/paper/2112.14710/) (2021)
- [A Reliable, Self-Adaptive Face Identification Framework via Lyapunov Optimization](https://mlsanity.com/paper/2109.01212/) (2021)
- [Tutorial on Course-of-Action (COA) Attack Search Methods in Computer Networks](https://mlsanity.com/paper/2205.13763/) (2022)
- [Spatio-Temporal Attack Course-of-Action (COA) Search Learning for Scalable and Time-Varying Networks](https://mlsanity.com/paper/2209.00862/) (2022)

## Abstract (arXiv)

This paper proposes a video delivery strategy for dynamic streaming services which maximizes time-average streaming quality under a playback delay constraint in wireless caching networks. The network where popular videos encoded by scalable video coding are already stored in randomly distributed caching nodes is considered under adaptive video streaming concepts, and distance-based interference management is investigated in this paper. In this network model, a streaming user makes delay-constrained decisions depending on stochastic network states: 1) caching node for video delivery, 2) video quality, and 3) the quantity of video chunks to receive. Since wireless link activation for video delivery may introduce delays, different timescales for updating caching node association, video quality adaptation, and chunk amounts are considered. After associating with a caching node for video delivery, the streaming user chooses combinations of quality and chunk amounts in the small timescale. The dynamic decision making process for video quality and chunk amounts at each slot is modeled using Markov decision process, and the caching node decision is made based on the framework of Lyapunov optimization. Our intensive simulations verify that the proposed video delivery algorithm works reliably and also can control the tradeoff between video quality and playback latency.