# Video compression dataset and benchmark of learning-based video-quality metrics

arXiv: https://arxiv.org/abs/2211.12109 · HTML: https://mlsanity.com/paper/2211.12109/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 54 references that are in the map.
- First posted on arXiv 22 Nov 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training](https://mlsanity.com/paper/2011.04263/) (2020)
- [Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment](https://mlsanity.com/paper/2008.03889/) (2020)
- [VideoSet: A Large-Scale Compressed Video Quality Dataset Based on JND Measurement](https://mlsanity.com/paper/1701.01500/) (2017)
- [ST-GREED: Space-Time Generalized Entropic Differences for Frame Rate Dependent Video Quality Prediction](https://mlsanity.com/paper/2010.13715/) (2020)
- [C3DVQA: Full-Reference Video Quality Assessment with 3D Convolutional Neural Network](https://mlsanity.com/paper/1910.13646/) (2019)
- [UGC-VIDEO: perceptual quality assessment of user-generated videos](https://mlsanity.com/paper/1908.11517/) (2019)
- [User-generated Video Quality Assessment: A Subjective and Objective Study](https://mlsanity.com/paper/2005.08527/) (2020)

## Built on by (most cited first, 27 of 27)

- [Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach](https://mlsanity.com/paper/2305.12726/) (2023)
- [Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks](https://mlsanity.com/paper/2310.06958/) (2023)
- [TSMD: A Database for Static Color Mesh Quality Assessment Study](https://mlsanity.com/paper/2308.01940/) (2023)
- [AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results](https://mlsanity.com/paper/2408.11982/) (2024)
- [NTIRE 2023 Quality Assessment of Video Enhancement Challenge](https://mlsanity.com/paper/2307.09729/) (2023)
- [TDMD: A Database for Dynamic Color Mesh Subjective and Objective Quality Explorations](https://mlsanity.com/paper/2308.01499/) (2023)
- [Adversarial purification for no-reference image-quality metrics: applicability study and new methods](https://mlsanity.com/paper/2404.06957/) (2024)
- [Guardians of Image Quality: Benchmarking Defenses Against Adversarial Attacks on Image Quality Metrics](https://mlsanity.com/paper/2408.01541/) (2024)
- [GeodesicPSIM: Predicting the Quality of Static Mesh with Texture Map via Geodesic Patch Similarity](https://mlsanity.com/paper/2308.04928/) (2023)
- [IOI: Invisible One-Iteration Adversarial Attack on No-Reference Image- and Video-Quality Metrics](https://mlsanity.com/paper/2403.05955/) (2024)
- [Ti-Patch: Tiled Physical Adversarial Patch for no-reference video quality metrics](https://mlsanity.com/paper/2404.09961/) (2024)
- [MVPBench: A Multi-Video Perception Evaluation Benchmark for Multi-Modal Video Understanding](https://mlsanity.com/paper/2603.22756/) (2026)
- [Exploring adversarial robustness of JPEG AI: methodology, comparison and new methods](https://mlsanity.com/paper/2411.11795/) (2024)
- [SR+Codec: a Benchmark of Super-Resolution for Video Compression Bitrate Reduction](https://mlsanity.com/paper/2305.04844/) (2023)
- [Learning on Bandwidth Constrained Multi-Source Data with MIMO-inspired DPP MAP Inference](https://mlsanity.com/paper/2306.02497/) (2023)
- [ViDA-UGC: Detailed Image Quality Analysis via Visual Distortion Assessment for UGC Images](https://mlsanity.com/paper/2508.12605/) (2025)
- [Applicability limitations of differentiable full-reference image-quality](https://mlsanity.com/paper/2212.05499/) (2022)
- [Subjective and Objective Quality-of-Experience Evaluation Study for Live Video Streaming](https://mlsanity.com/paper/2409.17596/) (2024)
- [AIM 2024 Challenge on Video Super-Resolution Quality Assessment: Methods and Results](https://mlsanity.com/paper/2410.04225/) (2024)
- [Machine vision-aware quality metrics for compressed image and video assessment](https://mlsanity.com/paper/2411.06776/) (2024)
- [CGVQM+D: Computer Graphics Video Quality Metric and Dataset](https://mlsanity.com/paper/2506.11546/) (2025)
- [NIC-RobustBench: A Comprehensive Open-Source Toolkit for Neural Image Compression and Robustness Analysis](https://mlsanity.com/paper/2506.19051/) (2025)
- [LEHA-CVQAD: Dataset To Enable Generalized Video Quality Assessment of Compression Artifacts](https://mlsanity.com/paper/2507.03990/) (2025)
- [CompSplat: Compression-aware 3D Gaussian Splatting for Real-world Video](https://mlsanity.com/paper/2602.09816/) (2026)
- [Application of Sensitivity Analysis Methods for Studying Neural Network Models](https://mlsanity.com/paper/2504.15100/) (2025)
- [Textured mesh Quality Assessment using Geometry and Color Field Similarity](https://mlsanity.com/paper/2505.10824/) (2025)
- [Learning Flexible Generalization in Video Quality Assessment by Bringing Device and Viewing Condition Distributions](https://mlsanity.com/paper/2607.04643/) (2026)

## Abstract (arXiv)

Video-quality measurement is a critical task in video processing. Nowadays, many implementations of new encoding standards - such as AV1, VVC, and LCEVC - use deep-learning-based decoding algorithms with perceptual metrics that serve as optimization objectives. But investigations of the performance of modern video- and image-quality metrics commonly employ videos compressed using older standards, such as AVC. In this paper, we present a new benchmark for video-quality metrics that evaluates video compression. It is based on a new dataset consisting of about 2,500 streams encoded using different standards, including AVC, HEVC, AV1, VP9, and VVC. Subjective scores were collected using crowdsourced pairwise comparisons. The list of evaluated metrics includes recent ones based on machine learning and neural networks. The results demonstrate that new no-reference metrics exhibit a high correlation with subjective quality and approach the capability of top full-reference metrics.