# MADRL-Based Rate Adaptation for 360deg Video Streaming with Multi-Viewpoint Prediction

arXiv: https://arxiv.org/abs/2405.07759 · HTML: https://mlsanity.com/paper/2405.07759/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 57 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Is Independent Learning All You Need in the StarCraft Multi-Agent Challenge?](https://mlsanity.com/paper/2011.09533/) (2020)
- [MANTRA: Memory Augmented Networks for Multiple Trajectory Prediction](https://mlsanity.com/paper/2006.03340/) (2020)
- [Predicting Head Movement in Panoramic Video: A Deep Reinforcement Learning Approach](https://mlsanity.com/paper/1710.10755/) (2017)
- [BOLA: Near-Optimal Bitrate Adaptation for Online Videos](https://mlsanity.com/paper/1601.06748/) (2016)
- [Adaptive 360 VR Video Streaming: Divide and Conquer!](https://mlsanity.com/paper/1609.08729/) (2016)
- [Sequential Reinforced 360-Degree Video Adaptive Streaming with Cross-user Attentive Network](https://mlsanity.com/paper/2009.13737/) (2020)

## Built on by (most cited first, 3 of 3)

- [Sphere-GAN: a GAN-based Approach for Saliency Estimation in 360deg Videos](https://mlsanity.com/paper/2509.11948/) (2025)
- [SalFormer360: a transformer-based saliency estimation model for 360-degree videos](https://mlsanity.com/paper/2602.04584/) (2026)
- [Transformer-Enhanced Reinforcement Learning: Fundamentals and Applications in Communication Networks](https://mlsanity.com/paper/2606.05208/) (2026)

## Abstract (arXiv)

Over the last few years, 360deg video traffic on the network has grown significantly. A key challenge of 360deg video playback is ensuring a high quality of experience (QoE) with limited network bandwidth. Currently, most studies focus on tile-based adaptive bitrate (ABR) streaming based on single viewport prediction to reduce bandwidth consumption. However, the performance of models for single-viewpoint prediction is severely limited by the inherent uncertainty in head movement, which can not cope with the sudden movement of users very well. This paper first presents a multimodal spatial-temporal attention transformer to generate multiple viewpoint trajectories with their probabilities given a historical trajectory. The proposed method models viewpoint prediction as a classification problem and uses attention mechanisms to capture the spatial and temporal characteristics of input video frames and viewpoint trajectories for multi-viewpoint prediction. After that, a multi-agent deep reinforcement learning (MADRL)-based ABR algorithm utilizing multi-viewpoint prediction for 360deg video streaming is proposed for maximizing different QoE objectives under various network conditions. We formulate the ABR problem as a decentralized partially observable Markov decision process (Dec-POMDP) problem and present a MAPPO algorithm based on centralized training and decentralized execution (CTDE) framework to solve the problem. The experimental results show that our proposed method improves the defined QoE metric by up to 85.5% compared to existing ABR methods.