# FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling

arXiv: https://arxiv.org/abs/2207.02595 · HTML: https://mlsanity.com/paper/2207.02595/

## Key facts

- Cited by 179 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 46 references that are in the map.
- First posted on arXiv 6 Jul 2022.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Video Swin Transformer](https://mlsanity.com/paper/2106.13230/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)
- [RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content](https://mlsanity.com/paper/2101.10955/) (2021)
- [Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perception](https://mlsanity.com/paper/2108.08505/) (2021)
- [Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training](https://mlsanity.com/paper/2011.04263/) (2020)
- [Learning Generalized Spatial-Temporal Deep Feature Representation for No-Reference Video Quality Assessment](https://mlsanity.com/paper/2012.13936/) (2020)
- [DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessment](https://mlsanity.com/paper/2206.09853/) (2022)
- [Subjective Quality Assessment for YouTube UGC Dataset](https://mlsanity.com/paper/2002.12275/) (2020)

## Built on by (most cited first, 30 of 179)

- [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023)
- [SkyReels-V2: Infinite-length Film Generative Model](https://mlsanity.com/paper/2504.13074/) (2025)
- [VideoScore: Building Automatic Metrics to Simulate Fine-grained Human Feedback for Video Generation](https://mlsanity.com/paper/2406.15252/) (2024)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Content](https://mlsanity.com/paper/2410.08260/) (2024)
- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [SeedVR: Seeding Infinity in Diffusion Transformer Towards Generic Video Restoration](https://mlsanity.com/paper/2501.01320/) (2025)
- [Fr'echet Video Motion Distance: A Metric for Evaluating Motion Consistency in Videos](https://mlsanity.com/paper/2407.16124/) (2024)
- [Towards A Better Metric for Text-to-Video Generation](https://mlsanity.com/paper/2401.07781/) (2024)
- [Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment](https://mlsanity.com/paper/2210.05357/) (2022)
- [MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos](https://mlsanity.com/paper/2303.14933/) (2023)
- [SWinGS: Sliding Windows for Dynamic 3D Gaussian Splatting](https://mlsanity.com/paper/2312.13308/) (2023)
- [Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach](https://mlsanity.com/paper/2305.12726/) (2023)
- [Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessment](https://mlsanity.com/paper/2403.11956/) (2024)
- [Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs](https://mlsanity.com/paper/2409.20063/) (2024)
- [Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Models](https://mlsanity.com/paper/2307.13981/) (2023)
- [KVQ: Kwai Video Quality Assessment for Short-form Videos](https://mlsanity.com/paper/2402.07220/) (2024)
- [PAI-Bench: A Comprehensive Benchmark For Physical AI](https://mlsanity.com/paper/2512.01989/) (2025)
- [GAIA: Rethinking Action Quality Assessment for AI-Generated Videos](https://mlsanity.com/paper/2406.06087/) (2024)
- [NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Results](https://mlsanity.com/paper/2404.11313/) (2024)
- [LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models](https://mlsanity.com/paper/2408.14008/) (2024)
- [FineVQ: Fine-Grained User Generated Content Video Quality Assessment](https://mlsanity.com/paper/2412.19238/) (2024)
- [Quality Assessment in the Era of Large Models: A Survey](https://mlsanity.com/paper/2409.00031/) (2024)
- [VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning](https://mlsanity.com/paper/2506.18564/) (2025)
- [Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessment](https://mlsanity.com/paper/2207.03723/) (2022)
- [AIGV-Assessor: Benchmarking and Evaluating the Perceptual Quality of Text-to-Video Generation with LMM](https://mlsanity.com/paper/2411.17221/) (2024)
- [Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterion](https://mlsanity.com/paper/2302.13269/) (2023)

## Abstract (arXiv)

Current deep video quality assessment (VQA) methods are usually with high computational costs when evaluating high-resolution videos. This cost hinders them from learning better video-quality-related representations via end-to-end training. Existing approaches typically consider naive sampling to reduce the computational cost, such as resizing and cropping. However, they obviously corrupt quality-related information in videos and are thus not optimal for learning good representations for VQA. Therefore, there is an eager need to design a new quality-retained sampling scheme for VQA. In this paper, we propose Grid Mini-patch Sampling (GMS), which allows consideration of local quality by sampling patches at their raw resolution and covers global quality with contextual relations via mini-patches sampled in uniform grids. These mini-patches are spliced and aligned temporally, named as fragments. We further build the Fragment Attention Network (FANet) specially designed to accommodate fragments as inputs. Consisting of fragments and FANet, the proposed FrAgment Sample Transformer for VQA (FAST-VQA) enables efficient end-to-end deep VQA and learns effective video-quality-related representations. It improves state-of-the-art accuracy by around 10% while reducing 99.5% FLOPs on 1080P high-resolution videos. The newly learned video-quality-related representations can also be transferred into smaller VQA datasets, boosting performance in these scenarios. Extensive experiments show that FAST-VQA has good performance on inputs of various resolutions while retaining high efficiency. We publish our code at https://github.com/timothyhtimothy/FAST-VQA.