# Predicting total time to compress a video corpus using online inference systems

arXiv: https://arxiv.org/abs/2410.18260 · HTML: https://mlsanity.com/paper/2410.18260/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 23 references that are in the map.
- First posted on arXiv 23 Oct 2024.

## Builds on (most cited first, 6 of 6)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Cost-Efficient and Robust On-Demand Video Transcoding Using Heterogeneous Cloud Services](https://mlsanity.com/paper/1711.01008/) (2017)
- [Content-adaptive Encoder Preset Prediction for Adaptive Live Streaming](https://mlsanity.com/paper/2210.10330/) (2022)
- [Filling the gaps in video transcoder deployment in the cloud](https://mlsanity.com/paper/2304.08634/) (2023)

## Abstract (arXiv)

Predicting the computational cost of compressing/transcoding clips in a video corpus is important for resource management of cloud services and VOD (Video On Demand) providers. Currently, customers of cloud video services are unaware of the cost of transcoding their files until the task is completed. Previous work concentrated on predicting perclip compression time, and thus estimating the cost of video compression. In this work, we propose new Machine Learning (ML) systems which predict cost for the entire corpus instead. This is a more appropriate goal since users are not interested in per-clip cost but instead the cost for the whole corpus. In this work, we evaluate our systems with respect to two video codecs (x264, x265) and a novel high-quality video corpus. We find that the accuracy of aggregate time prediction for a video corpus more than two times better than using per-clip predictions. Furthermore, we present an online inference framework in which we update the ML models as files are processed. A consideration of video compute overhead and appropriate choice of ML predictor for each fraction of corpus completed yields a prediction error of less than 5%. This is approximately two times better than previous work which proposed generalised predictors.