# Convex Hull Prediction for Adaptive Video Streaming by Recurrent Learning

arXiv: https://arxiv.org/abs/2206.04877 · HTML: https://mlsanity.com/paper/2206.04877/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 60 references that are in the map.
- First posted on arXiv 10 Jun 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Delving Deeper into Convolutional Networks for Learning Video Representations](https://mlsanity.com/paper/1511.06432/) (2015)
- [BVI-DVC: A Training Database for Deep Video Compression](https://mlsanity.com/paper/2003.13552/) (2020)
- [AdaPool: Exponential Adaptive Pooling for Information-Retaining Downsampling](https://mlsanity.com/paper/2111.00772/) (2021)
- [VideoSet: A Large-Scale Compressed Video Quality Dataset Based on JND Measurement](https://mlsanity.com/paper/1701.01500/) (2017)
- [Recurrent Fully Convolutional Networks for Video Segmentation](https://mlsanity.com/paper/1606.00487/) (2016)
- [Efficient Bitrate Ladder Construction for Content-Optimized Adaptive Video Streaming](https://mlsanity.com/paper/2102.04550/) (2021)
- [VMAF-based Bitrate Ladder Estimation for Adaptive Streaming](https://mlsanity.com/paper/2103.07564/) (2021)
- [A GMM-Based Stair Quality Model for Human Perceived JPEG Images](https://mlsanity.com/paper/1511.03398/) (2015)

## Built on by (most cited first, 4 of 4)

- [Leveraging Compression to Construct Transferable Bitrate Ladders](https://mlsanity.com/paper/2512.12952/) (2025)
- [Efficient Bitrate Ladder Construction using Transfer Learning and Spatio-Temporal Features](https://mlsanity.com/paper/2401.03195/) (2024)
- [Optimal Transcoding Resolution Prediction for Efficient Per-Title Bitrate Ladder Estimation](https://mlsanity.com/paper/2401.04405/) (2024)
- [Content Adaptive Encoding For Interactive Game Streaming](https://mlsanity.com/paper/2511.22327/) (2025)

## Abstract (arXiv)

Adaptive video streaming relies on the construction of efficient bitrate ladders to deliver the best possible visual quality to viewers under bandwidth constraints. The traditional method of content dependent bitrate ladder selection requires a video shot to be pre-encoded with multiple encoding parameters to find the optimal operating points given by the convex hull of the resulting rate-quality curves. However, this pre-encoding step is equivalent to an exhaustive search process over the space of possible encoding parameters, which causes significant overhead in terms of both computation and time expenditure. To reduce this overhead, we propose a deep learning based method of content aware convex hull prediction. We employ a recurrent convolutional network (RCN) to implicitly analyze the spatiotemporal complexity of video shots in order to predict their convex hulls. A two-step transfer learning scheme is adopted to train our proposed RCN-Hull model, which ensures sufficient content diversity to analyze scene complexity, while also making it possible to capture the scene statistics of pristine source videos. Our experimental results reveal that our proposed model yields better approximations of the optimal convex hulls, and offers competitive time savings as compared to existing approaches. On average, the pre-encoding time was reduced by 53.8% by our method, while the average Bjontegaard delta bitrate (BD-rate) of the predicted convex hulls against ground truth was 0.26%, and the mean absolute deviation of the BD-rate distribution was 0.57%.