# Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models

arXiv: https://arxiv.org/abs/2307.13981 · HTML: https://mlsanity.com/paper/2307.13981/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 90 references that are in the map.
- First posted on arXiv 26 Jul 2023.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [InceptionTime: Finding AlexNet for Time Series Classification](https://mlsanity.com/paper/1909.04939/) (2019)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [Massive Online Crowdsourced Study of Subjective and Objective Picture Quality](https://mlsanity.com/paper/1511.02919/) (2015)
- [FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling](https://mlsanity.com/paper/2207.02595/) (2022)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild](https://mlsanity.com/paper/2005.13983/) (2020)
- [A Deep Learning based No-reference Quality Assessment Model for UGC Videos](https://mlsanity.com/paper/2204.14047/) (2022)

## Built on by (most cited first, 30 of 38)

- [NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results](https://mlsanity.com/paper/2404.11313/) (2024)
- [LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models](https://mlsanity.com/paper/2408.14008/) (2024)
- [FineVQ: Fine-Grained User Generated Content Video Quality Assessment](https://mlsanity.com/paper/2412.19238/) (2024)
- [VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning](https://mlsanity.com/paper/2506.18564/) (2025)
- [Modular Blind Video Quality Assessment](https://mlsanity.com/paper/2402.19276/) (2024)
- [Enhancing Blind Video Quality Assessment with Rich Quality-aware Features](https://mlsanity.com/paper/2405.08745/) (2024)
- [VQA2: Visual Question Answering for Video Quality Assessment](https://mlsanity.com/paper/2411.03795/) (2024)
- [NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Results](https://mlsanity.com/paper/2504.13131/) (2025)
- [Benchmarking Multi-dimensional AIGC Video Quality Assessment: A Dataset and Unified Model](https://mlsanity.com/paper/2407.21408/) (2024)
- [FVQ: A Large-Scale Dataset and an LMM-based Method for Face Video Quality Assessment](https://mlsanity.com/paper/2504.09255/) (2025)
- [VQAThinker: Exploring Generalizable and Explainable Video Quality Assessment via Reinforcement Learning](https://mlsanity.com/paper/2508.06051/) (2025)
- [Assessing UHD Image Quality from Aesthetics, Distortions, and Saliency](https://mlsanity.com/paper/2409.00749/) (2024)
- [Human-Activity AGV Quality Assessment: A Benchmark Dataset and an Objective Evaluation Metric](https://mlsanity.com/paper/2411.16619/) (2024)
- [Dual-Branch Network for Portrait Image Quality Assessment](https://mlsanity.com/paper/2405.08555/) (2024)
- [Generalizable Video Quality Assessment via Weak-to-Strong Learning](https://mlsanity.com/paper/2505.03631/) (2025)
- [Scaling-up Perceptual Video Quality Assessment](https://mlsanity.com/paper/2505.22543/) (2025)
- [CompressedVQA-HDR: Generalized Full-reference and No-reference Quality Assessment Models for Compressed High Dynamic Range Videos](https://mlsanity.com/paper/2507.11900/) (2025)
- [AIS 2024 Challenge on Video Quality Assessment of User-Generated Content: Methods and Results](https://mlsanity.com/paper/2404.16205/) (2024)
- [An Ensemble Approach to Short-form Video Quality Assessment Using Multimodal LLM](https://mlsanity.com/paper/2412.18060/) (2024)
- [Video Quality Assessment for Online Processing: From Spatial to Temporal Sampling](https://mlsanity.com/paper/2501.07087/) (2025)
- [Engagement Prediction of Short Videos with Large Multimodal Models](https://mlsanity.com/paper/2508.02516/) (2025)
- [SG-JND: Semantic-Guided Just Noticeable Distortion Predictor For Image Compression](https://mlsanity.com/paper/2408.04273/) (2024)
- [QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding](https://mlsanity.com/paper/2601.18195/) (2026)
- [FreeSpec: Training-Free Long Video Generation via Singular-Spectrum Reconstruction](https://mlsanity.com/paper/2605.06509/) (2026)
- [Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model](https://mlsanity.com/paper/2607.16742/) (2026)
- [Computational Analysis of Degradation Modeling in Blind Panoramic Image Quality Assessment](https://mlsanity.com/paper/2503.03255/) (2025)
- [Context and Pixel Aware Large Language Model for Video Quality Assessment](https://mlsanity.com/paper/2505.16025/) (2025)
- [VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment](https://mlsanity.com/paper/2511.17962/) (2025)
- [Convolutions Need Registers Too: HVS-Inspired Dynamic Attention for Video Quality Assessment](https://mlsanity.com/paper/2601.11045/) (2026)
- [MDS-VQA: Model-Informed Data Selection for Video Quality Assessment](https://mlsanity.com/paper/2603.11525/) (2026)

## Abstract (arXiv)

Blind video quality assessment (BVQA) plays an indispensable role in monitoring and improving the end-users' viewing experience in various real-world video-enabled media applications. As an experimental field, the improvements of BVQA models have been measured primarily on a few human-rated VQA datasets. Thus, it is crucial to gain a better understanding of existing VQA datasets in order to properly evaluate the current progress in BVQA. Towards this goal, we conduct a first-of-its-kind computational analysis of VQA datasets via designing minimalistic BVQA models. By minimalistic, we restrict our family of BVQA models to build only upon basic blocks: a video preprocessor (for aggressive spatiotemporal downsampling), a spatial quality analyzer, an optional temporal quality analyzer, and a quality regressor, all with the simplest possible instantiations. By comparing the quality prediction performance of different model variants on eight VQA datasets with realistic distortions, we find that nearly all datasets suffer from the easy dataset problem of varying severity, some of which even admit blind image quality assessment (BIQA) solutions. We additionally justify our claims by contrasting our model generalizability on these VQA datasets, and by ablating a dizzying set of BVQA design choices related to the basic building blocks. Our results cast doubt on the current progress in BVQA, and meanwhile shed light on good practices of constructing next-generation VQA datasets and models.