Structured Reinforcement Learning for Media Streaming at the Wireless Edge
On the map: Deep reinforcement learning methods → RL for energy, networks and operations
Abstract (arXiv)
Media streaming is the dominant application over wireless edge (access) networks. The increasing softwarization of such networks has led to efforts at intelligent control, wherein application-specific actions may be dynamically taken to enhance the user experience. The goal of this work is to develop and demonstrate learning-based policies for optimal decision making to determine which clients to dynamically prioritize in a video streaming setting. We formulate the policy design question as a constrained Markov decision problem (CMDP), and observe that by using a Lagrangian relaxation we can decompose it into single-client problems. Further, the optimal policy takes a threshold form in the video buffer length, which enables us to design an efficient constrained reinforcement learning (CRL) algorithm to learn it. Specifically, we show that a natural policy gradient (NPG) based algorithm that is derived using the structure of our problem converges to the globally optimal policy. We then develop a simulation environment for training, and a real-world intelligent controller attached to a WiFi access point for evaluation. We empirically show that the structured learning approach enables fast learning. Furthermore, such a structured policy can be easily deployed due to low computational complexity, leading to policy execution taking only about 15μs. Using YouTube streaming experiments in a resource constrained scenario, we demonstrate that the CRL approach can increase quality of experience (QOE) by over 30%.
Key facts. Cited by 1 paper in the map (data through 13 Oct 2026). Placed in RL for energy, networks and operations, one of 15 directions in Deep reinforcement learning methods. Builds on 13 of its 48 references that are in the map. First posted on arXiv 10 Apr 2024.
Builds on
13 of 48 references.
References · 13
- 2017Proximal Policy Optimization Algorithmscited by 14,592
- 2017Constrained Policy Optimizationcited by 931
- 2020Exploration-Exploitation in Constrained MDPscited by 147
- 2019Constrained Reinforcement Learning Has Zero Duality Gapcited by 144
- 2020Provably Efficient Safe Exploration via Primal-Dual Policy Optimizationcited by 132
- 2021Learning Policies with Zero or Bounded Constraint Violation for Constrained MDPscited by 79
- 2022Near-Optimal Sample Complexity Bounds for Constrained MDPscited by 37
- 2021DOPE: Doubly Optimistic and Pessimistic Exploration for Safe Reinforcement Learningcited by 35
- 2020A Sample-Efficient Algorithm for Episodic Finite-Horizon MDP with Constraintscited by 34
- 2020Learning with Safety Constraints: Sample Complexity of Reinforcement Learning for Constrained MDPscited by 32
- 2019Online Reinforcement Learning of Optimal Threshold Policies for Markov Decision Processescited by 7
- 2018Inexact First-Order Primal-Dual Algorithmscited by 6
Built on by
Citing papers per year: 2025: 1.
1 paper.