# Zoom-VQA: Patches, Frames and Clips Integration for Video Quality Assessment

arXiv: https://arxiv.org/abs/2304.06440 · HTML: https://mlsanity.com/paper/2304.06440/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 60 references that are in the map.
- First posted on arXiv 13 Apr 2023.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [SimMIM: A Simple Framework for Masked Image Modeling](https://mlsanity.com/paper/2111.09886/) (2021)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classification](https://mlsanity.com/paper/1712.04851/) (2017)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [Learning Spatio-Temporal Representation with Pseudo-3D Residual Networks](https://mlsanity.com/paper/1711.10305/) (2017)
- [Incorporating Convolution Designs into Visual Transformers](https://mlsanity.com/paper/2103.11816/) (2021)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling](https://mlsanity.com/paper/2207.02595/) (2022)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [A Deep Learning based No-reference Quality Assessment Model for UGC Videos](https://mlsanity.com/paper/2204.14047/) (2022)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)

## Built on by (most cited first, 21 of 21)

- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [NTIRE 2024 Challenge on Image Super-Resolution (x4): Methods and Results](https://mlsanity.com/paper/2404.09790/) (2024)
- [KVQ: Kwai Video Quality Assessment for Short-form Videos](https://mlsanity.com/paper/2402.07220/) (2024)
- [NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results](https://mlsanity.com/paper/2404.11313/) (2024)
- [Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video Quality Assessment](https://mlsanity.com/paper/2308.00729/) (2023)
- [Capturing Co-existing Distortions in User-Generated Content for No-reference Video Quality Assessment](https://mlsanity.com/paper/2307.16813/) (2023)
- [CPGA: Coding Priors-Guided Aggregation Network for Compressed Video Quality Enhancement](https://mlsanity.com/paper/2403.10362/) (2024)
- [CLiF-VQA: Enhancing Video Quality Assessment by Incorporating High-Level Semantic Information related to Human Feelings](https://mlsanity.com/paper/2311.07090/) (2023)
- [KVQ: Boosting Video Quality Assessment via Saliency-guided Local Perception](https://mlsanity.com/paper/2503.10259/) (2025)
- [NTIRE 2023 Quality Assessment of Video Enhancement Challenge](https://mlsanity.com/paper/2307.09729/) (2023)
- [ReLaX-VQA: Residual Fragment and Layer Stack Extraction for Enhancing Video Quality Assessment](https://mlsanity.com/paper/2407.11496/) (2024)
- [Video Quality Assessment for Online Processing: From Spatial to Temporal Sampling](https://mlsanity.com/paper/2501.07087/) (2025)
- [A New Dataset and Framework for Real-World Blurred Images Super-Resolution](https://mlsanity.com/paper/2407.14880/) (2024)
- [OAPT: Offset-Aware Partition Transformer for Double JPEG Artifacts Removal](https://mlsanity.com/paper/2408.11480/) (2024)
- [Multilevel Semantic-Aware Model for AI-Generated Video Quality Assessment](https://mlsanity.com/paper/2501.02706/) (2025)
- [MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment](https://mlsanity.com/paper/2504.16003/) (2025)
- [DIVA-VQA: Detecting Inter-frame Variations in UGC Video Quality](https://mlsanity.com/paper/2508.10605/) (2025)
- [CAMP-VQA: Caption-Embedded Multimodal Perception for No-Reference Quality Assessment of Compressed Video](https://mlsanity.com/paper/2511.07290/) (2025)
- [Head-Aware Visual Cropping: Enhancing Fine-Grained VQA with Attention-Guided Subimage](https://mlsanity.com/paper/2601.22483/) (2026)
- [Secure Video Quality Assessment Resisting Adversarial Attacks](https://mlsanity.com/paper/2410.06866/) (2024)
- [High-Fidelity Video Quality Assessment with VQA-Specific Saliency](https://mlsanity.com/paper/2609.16946/) (2026)

## Abstract (arXiv)

Video quality assessment (VQA) aims to simulate the human perception of video quality, which is influenced by factors ranging from low-level color and texture details to high-level semantic content. To effectively model these complicated quality-related factors, in this paper, we decompose video into three levels (ie, patch level, frame level, and clip level), and propose a novel Zoom-VQA architecture to perceive spatio-temporal features at different levels. It integrates three components: patch attention module, frame pyramid alignment, and clip ensemble strategy, respectively for capturing region-of-interest in the spatial dimension, multi-level information at different feature levels, and distortions distributed over the temporal dimension. Owing to the comprehensive design, Zoom-VQA obtains state-of-the-art results on four VQA benchmarks and achieves 2nd place in the NTIRE 2023 VQA challenge. Notably, Zoom-VQA has outperformed the previous best results on two subsets of LSVQ, achieving 0.8860 (+1.0%) and 0.7985 (+1.9%) of SRCC on the respective subsets. Adequate ablation studies further verify the effectiveness of each component. Codes and models are released in https://github.com/k-zha14/Zoom-VQA.