# CNN-based Prediction of Partition Path for VVC Fast Inter Partitioning Using Motion Fields

arXiv: https://arxiv.org/abs/2310.13838 · HTML: https://mlsanity.com/paper/2310.13838/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 10 of its 43 references that are in the map.
- First posted on arXiv 20 Oct 2023.

## Builds on (most cited first, 10 of 10)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [BVI-DVC: A Training Database for Deep Video Compression](https://mlsanity.com/paper/2003.13552/) (2020)
- [An Efficient Four-Parameter Affine Motion Model for Video Coding](https://mlsanity.com/paper/1702.06297/) (2017)
- [DeepQTMT: A Deep Learning Approach for Fast QTMT-based CU Partition of Intra-mode VVC](https://mlsanity.com/paper/2006.13125/) (2020)
- [Learned Fast HEVC Intra Coding](https://mlsanity.com/paper/1907.02287/) (2019)
- [CNN-based driving of block partitioning for intra slices encoding](https://mlsanity.com/paper/2011.06691/) (2020)
- [Machine Learning based Efficient QT-MTT Partitioning Scheme for VVC Intra Encoders](https://mlsanity.com/paper/2103.05319/) (2021)

## Abstract (arXiv)

The Versatile Video Coding (VVC) standard has been recently finalized by the Joint Video Exploration Team (JVET). Compared to the High Efficiency Video Coding (HEVC) standard, VVC offers about 50% compression efficiency gain, in terms of Bjontegaard Delta-Rate (BD-rate), at the cost of a 10-fold increase in encoding complexity. In this paper, we propose a method based on Convolutional Neural Network (CNN) to speed up the inter partitioning process in VVC. Firstly, a novel representation for the quadtree with nested multi-type tree (QTMT) partition is introduced, derived from the partition path. Secondly, we develop a U-Net-based CNN taking a multi-scale motion vector field as input at the Coding Tree Unit (CTU) level. The purpose of CNN inference is to predict the optimal partition path during the Rate-Distortion Optimization (RDO) process. To achieve this, we divide CTU into grids and predict the Quaternary Tree (QT) depth and Multi-type Tree (MT) split decisions for each cell of the grid. Thirdly, an efficient partition pruning algorithm is introduced to employ the CNN predictions at each partitioning level to skip RDO evaluations of unnecessary partition paths. Finally, an adaptive threshold selection scheme is designed, making the trade-off between complexity and efficiency scalable. Experiments show that the proposed method can achieve acceleration ranging from 16.5% to 60.2% under the RandomAccess Group Of Picture 32 (RAGOP32) configuration with a reasonable efficiency drop ranging from 0.44% to 4.59% in terms of BD-rate, which surpasses other state-of-the-art solutions. Additionally, our method stands out as one of the lightest approaches in the field, which ensures its applicability to other encoders.