# A Deep Learning based No-reference Quality Assessment Model for UGC Videos

arXiv: https://arxiv.org/abs/2204.14047 · HTML: https://mlsanity.com/paper/2204.14047/

## Key facts

- Cited by 129 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 29 Apr 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild](https://mlsanity.com/paper/2005.13983/) (2020)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content](https://mlsanity.com/paper/2101.10955/) (2021)
- [Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perception](https://mlsanity.com/paper/2108.08505/) (2021)
- [Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training](https://mlsanity.com/paper/2011.04263/) (2020)
- [Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature Fusion and Iterative Mixed Database Training](https://mlsanity.com/paper/2105.14550/) (2021)
- [Continual Learning for Blind Image Quality Assessment](https://mlsanity.com/paper/2102.09717/) (2021)
- [Deep Learning based Full-reference and No-reference Quality Assessment Models for Compressed UGC Videos](https://mlsanity.com/paper/2106.01111/) (2021)
- [Subjective and Objective Quality Assessment of High Frame Rate Videos](https://mlsanity.com/paper/2007.11634/) (2020)
- [A Comparative Evaluation of Temporal Pooling Methods for Blind Video Quality Assessment](https://mlsanity.com/paper/2002.10651/) (2020)
- [Task-Specific Normalization for Continual Learning of Blind Image Quality Models](https://mlsanity.com/paper/2107.13429/) (2021)
- [A strong baseline for image and video quality assessment](https://mlsanity.com/paper/2111.07104/) (2021)
- [Subjective and Objective Analysis of Streamed Gaming Videos](https://mlsanity.com/paper/2203.12824/) (2022)
- [FAVER: Blind Quality Prediction of Variable Frame Rate Videos](https://mlsanity.com/paper/2201.01492/) (2022)

## Built on by (most cited first, 30 of 129)

- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [Bilateral Reference for High-Resolution Dichotomous Image Segmentation](https://mlsanity.com/paper/2401.03407/) (2024)
- [AI-Generated Content (AIGC): A Survey](https://mlsanity.com/paper/2304.06632/) (2023)
- [Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos](https://mlsanity.com/paper/2407.16124/) (2024)
- [MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos](https://mlsanity.com/paper/2303.14933/) (2023)
- [Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach](https://mlsanity.com/paper/2305.12726/) (2023)
- [Perceptual Video Quality Assessment: A Survey](https://mlsanity.com/paper/2402.03413/) (2024)
- [DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessment](https://mlsanity.com/paper/2206.09853/) (2022)
- [MM-PCQA: Multi-Modal Learning for No-reference Point Cloud Quality Assessment](https://mlsanity.com/paper/2209.00244/) (2022)
- [Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment](https://mlsanity.com/paper/2403.11956/) (2024)
- [Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs](https://mlsanity.com/paper/2409.20063/) (2024)
- [Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models](https://mlsanity.com/paper/2307.13981/) (2023)
- [KVQ: Kwai Video Quality Assessment for Short-form Videos](https://mlsanity.com/paper/2402.07220/) (2024)
- [GAIA: Rethinking Action Quality Assessment for AI-Generated Videos](https://mlsanity.com/paper/2406.06087/) (2024)
- [NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results](https://mlsanity.com/paper/2404.11313/) (2024)
- [LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models](https://mlsanity.com/paper/2408.14008/) (2024)
- [FineVQ: Fine-Grained User Generated Content Video Quality Assessment](https://mlsanity.com/paper/2412.19238/) (2024)
- [A No-reference Quality Assessment Metric for Point Cloud Based on Captured Video Sequences](https://mlsanity.com/paper/2206.05054/) (2022)
- [AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM](https://mlsanity.com/paper/2411.17221/) (2024)
- [Modular Blind Video Quality Assessment](https://mlsanity.com/paper/2402.19276/) (2024)
- [Evaluating Point Cloud from Moving Camera Videos: A No-Reference Metric](https://mlsanity.com/paper/2208.14085/) (2022)
- [Zoom-VQA: Patches, Frames and Clips Integration for Video Quality Assessment](https://mlsanity.com/paper/2304.06440/) (2023)
- [GMS-3DQA: Projection-based Grid Mini-patch Sampling for 3D Model Quality Assessment](https://mlsanity.com/paper/2306.05658/) (2023)
- [NTIRE 2024 Quality Assessment of AI-Generated Content Challenge](https://mlsanity.com/paper/2404.16687/) (2024)
- [Enhancing Blind Video Quality Assessment with Rich Quality-aware Features](https://mlsanity.com/paper/2405.08745/) (2024)
- [Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessment](https://mlsanity.com/paper/2304.14672/) (2023)
- [StableVQA: A Deep No-Reference Quality Assessment Model for Video Stability](https://mlsanity.com/paper/2308.04904/) (2023)
- [VQA2: Visual Question Answering for Video Quality Assessment](https://mlsanity.com/paper/2411.03795/) (2024)
- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)

## Abstract (arXiv)

Quality assessment for User Generated Content (UGC) videos plays an important role in ensuring the viewing experience of end-users. Previous UGC video quality assessment (VQA) studies either use the image recognition model or the image quality assessment (IQA) models to extract frame-level features of UGC videos for quality regression, which are regarded as the sub-optimal solutions because of the domain shifts between these tasks and the UGC VQA task. In this paper, we propose a very simple but effective UGC VQA model, which tries to address this problem by training an end-to-end spatial feature extraction network to directly learn the quality-aware spatial feature representation from raw pixels of the video frames. We also extract the motion features to measure the temporal-related distortions that the spatial features cannot model. The proposed model utilizes very sparse frames to extract spatial features and dense frames (i.e. the video chunk) with a very low spatial resolution to extract motion features, which thereby has low computational complexity. With the better quality-aware features, we only use the simple multilayer perception layer (MLP) network to regress them into the chunk-level quality scores, and then the temporal average pooling strategy is adopted to obtain the video-level quality score. We further introduce a multi-scale quality fusion strategy to solve the problem of VQA across different spatial resolutions, where the multi-scale weights are obtained from the contrast sensitivity function of the human visual system. The experimental results show that the proposed model achieves the best performance on five popular UGC VQA databases, which demonstrates the effectiveness of the proposed model. The code will be publicly available.