# VMAF And Variants: Towards A Unified VQA

arXiv: https://arxiv.org/abs/2103.07770 · HTML: https://mlsanity.com/paper/2103.07770/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 24 references that are in the map.
- First posted on arXiv 13 Mar 2021.

## Builds on (most cited first, 8 of 8)

- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training](https://mlsanity.com/paper/2011.04263/) (2020)
- [SpatioTemporal Feature Integration and Model Fusion for Full Reference Video Quality Assessment](https://mlsanity.com/paper/1804.04813/) (2018)
- [AQuA: Analytical Quality Assessment for Optimizing Video Analytics Systems](https://mlsanity.com/paper/2101.09752/) (2021)
- [Regression or Classification? New Methods to Evaluate No-Reference Picture and Video Quality Models](https://mlsanity.com/paper/2102.00155/) (2021)

## Built on by (most cited first, 2 of 2)

- [A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment](https://mlsanity.com/paper/2403.10854/) (2024)
- [Full Reference Video Quality Assessment for Machine Learning-Based Video Codecs](https://mlsanity.com/paper/2309.00769/) (2023)

## Abstract (arXiv)

Video quality assessment (VQA) is now a fast-growing subject, maturing in the full reference (FR) case, yet challenging in the exploding no reference (NR) case. We investigate variants of the popular VMAF video quality assessment algorithm for the FR case, using both support vector regression and feedforward neural networks. We extend it to the NR case, using some different features but similar learning, to develop a partially unified framework for VQA. When fully trained, FR algorithms such as VMAF perform very well on test datasets, reaching 90%+ match in PCC and SRCC; but for predicting performance in the wild, we train/test from scratch for each database. With an 80/20 train/test split, we still achieve about 90% performance on average in both PCC and SRCC, with up to 7-9% gains over VMAF, using an improved motion feature and better regression. Moreover, we even get decent performance (about 75%) if we ignore the reference, treating FR as NR, partly justifying our attempts at unification. In the true NR case, we reduce complexity vs. leading recent algorithms VIDEVAL, RAPIQUE, yet achieve performance within 3-5%. Moreover, we develop a method to analyze the saliency of features, and conclude that for both VIDEVAL and RAPIQUE, a small subset of their features are providing the bulk of the performance. In short, we find encouraging improvements in trainability in FR, while constraining training complexity against leading methods in NR, elucidating the saliency of features for feature selection.