# Capturing Co-existing Distortions in User-Generated Content for No-reference Video Quality Assessment

arXiv: https://arxiv.org/abs/2307.16813 · HTML: https://mlsanity.com/paper/2307.16813/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 73 references that are in the map.
- First posted on arXiv 31 Jul 2023.

## Builds on (most cited first, 30 of 37)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutions](https://mlsanity.com/paper/2102.12122/) (2021)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [Video Classification with Channel-Separated Convolutional Networks](https://mlsanity.com/paper/1904.02811/) (2019)
- [Incorporating Convolution Designs into Visual Transformers](https://mlsanity.com/paper/2103.11816/) (2021)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content](https://mlsanity.com/paper/2101.10955/) (2021)
- [Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perception](https://mlsanity.com/paper/2108.08505/) (2021)
- [Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training](https://mlsanity.com/paper/2011.04263/) (2020)
- [Video Representation Learning with Visual Tempo Consistency](https://mlsanity.com/paper/2006.15489/) (2020)

## Built on by (most cited first, 18 of 18)

- [Perceptual Video Quality Assessment: A Survey](https://mlsanity.com/paper/2402.03413/) (2024)
- [XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolution](https://mlsanity.com/paper/2403.05049/) (2024)
- [KVQ: Kwai Video Quality Assessment for Short-form Videos](https://mlsanity.com/paper/2402.07220/) (2024)
- [NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results](https://mlsanity.com/paper/2404.11313/) (2024)
- [FineVQ: Fine-Grained User Generated Content Video Quality Assessment](https://mlsanity.com/paper/2412.19238/) (2024)
- [CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2403.10362/) (2024)
- [PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild](https://mlsanity.com/paper/2405.17765/) (2024)
- [FVQ: A Large-Scale Dataset and an LMM-based Method for Face Video Quality Assessment](https://mlsanity.com/paper/2504.09255/) (2025)
- [KVQ: Boosting Video Quality Assessment via Saliency-guided Local Perception](https://mlsanity.com/paper/2503.10259/) (2025)
- [QPT V2: Masked Image Modeling Advances Visual Scoring](https://mlsanity.com/paper/2407.16541/) (2024)
- [CasSR: Activating Image Power for Real-World Image Super-Resolution](https://mlsanity.com/paper/2403.11451/) (2024)
- [InternVQA: Advancing Compressed Video Quality Assessment with Distilling Large Foundation Model](https://mlsanity.com/paper/2502.19026/) (2025)
- [A New Dataset and Framework for Real-World Blurred Images Super-Resolution](https://mlsanity.com/paper/2407.14880/) (2024)
- [OAPT: Offset-Aware Partition Transformer for Double JPEG Artifacts Removal](https://mlsanity.com/paper/2408.11480/) (2024)
- [MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessment](https://mlsanity.com/paper/2509.11589/) (2025)
- [Score2Instruct: Scaling Up Video Quality-Centric Instructions via Automated Dimension Scoring](https://mlsanity.com/paper/2506.21011/) (2025)
- [Pioneering Perceptual Video Fluency Assessment: A Novel Task with Benchmark Dataset and Baseline](https://mlsanity.com/paper/2603.26055/) (2026)
- [FGSVQA: Frequency-Guided Short-form Video Quality Assessment](https://mlsanity.com/paper/2605.20016/) (2026)

## Abstract (arXiv)

Video Quality Assessment (VQA), which aims to predict the perceptual quality of a video, has attracted raising attention with the rapid development of streaming media technology, such as Facebook, TikTok, Kwai, and so on. Compared with other sequence-based visual tasks (textite.g., action recognition), VQA faces two under-estimated challenges unresolved in User Generated Content (UGC) videos. textitFirst, it is not rare that several frames containing serious distortions (textite.g.,blocking, blurriness), can determine the perceptual quality of the whole video, while other sequence-based tasks require more frames of equal importance for representations. textitSecond, the perceptual quality of a video exhibits a multi-distortion distribution, due to the differences in the duration and probability of occurrence for various distortions. In order to solve the above challenges, we propose textitVisual Quality Transformer (VQT) to extract quality-related sparse features more efficiently. Methodologically, a Sparse Temporal Attention (STA) is proposed to sample keyframes by analyzing the temporal correlation between frames, which reduces the computational complexity from O(T2) to O(T log T). Structurally, a Multi-Pathway Temporal Network (MPTN) utilizes multiple STA modules with different degrees of sparsity in parallel, capturing co-existing distortions in a video. Experimentally, VQT demonstrates superior performance than many textitstate-of-the-art methods in three public no-reference VQA datasets. Furthermore, VQT shows better performance in four full-reference VQA datasets against widely-adopted industrial algorithms (textiti.e., VMAF and AVQT).