Patch-VQ: 'Patching Up' the Video Quality Problem
Not on the map. Why?
Abstract (arXiv)
No-reference (NR) perceptual video quality assessment (VQA) is a complex, unsolved, and important problem to social and streaming media applications. Efficient and accurate video quality predictors are needed to monitor and guide the processing of billions of shared, often imperfect, user-generated content (UGC). Unfortunately, current NR models are limited in their prediction capabilities on real-world, "in-the-wild" UGC video data. To advance progress on this problem, we created the largest (by far) subjective video quality dataset, containing 39, 000 realworld distorted videos and 117, 000 space-time localized video patches ('v-patches'), and 5.5M human perceptual quality annotations. Using this, we created two unique NR-VQA models: (a) a local-to-global region-based NR VQA architecture (called PVQ) that learns to predict global video quality and achieves state-of-the-art performance on 3 UGC datasets, and (b) a first-of-a-kind space-time video quality mapping engine (called PVQ Mapper) that helps localize and visualize perceptual distortions in space and time. We will make the new database and prediction models available immediately following the review process.
Key facts. Cited by 147 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Nov 2020.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 4; 2022: 16; 2023: 25; 2024: 42; 2025: 37; 2026: 23.
147 papers.
Sort:
By area:
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
- 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
- 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
- 2022A Deep Learning based No-reference Quality Assessment Model for UGC Videoscited by 128
- 2021RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Contentcited by 100
- 2021Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perceptioncited by 97
- 2022Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessmentcited by 50
- 2024Towards A Better Metric for Text-to-Video Generationcited by 50
- 2023MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videoscited by 46
- 2023Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approachcited by 45
- 2022DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessmentcited by 41
38 more
- 2024Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMscited by 39
- 2023Analysis of Video Quality Datasets via Design of Minimalistic Video Quality Modelscited by 38
- 2024KVQ: Kwai Video Quality Assessment for Short-form Videoscited by 38
- 2024GAIA: Rethinking Action Quality Assessment for AI-Generated Videoscited by 30
- 2024NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Resultscited by 29
- 2024LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Modelscited by 28
- 2024FineVQ: Fine-Grained User Generated Content Video Quality Assessmentcited by 28
- 2022Video compression dataset and benchmark of learning-based video-quality metricscited by 27
- 2023Advancing Zero-Shot Digital Human Quality Assessment through Text-Prompted Evaluationcited by 27
- 2024Quality Assessment in the Era of Large Models: A Surveycited by 26
- 2025VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learningcited by 26
- 2022Exploring the Effectiveness of Video Perceptual Representation in Blind Video Quality Assessmentcited by 25
- 2023Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterioncited by 23
- 2024Modular Blind Video Quality Assessmentcited by 23
- 2023Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Imagescited by 21
- 2023Zoom-VQA: Patches, Frames and Clips Integration for Video Quality Assessmentcited by 21
- 2023Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video Quality Assessmentcited by 20
- 2024NTIRE 2024 Quality Assessment of AI-Generated Content Challengecited by 20
- 2022CONVIQT: Contrastive Video Quality Estimatorcited by 19
- 2024Enhancing Blind Video Quality Assessment with Rich Quality-aware Featurescited by 19
- 2023Capturing Co-existing Distortions in User-Generated Content for No-reference Video Quality Assessmentcited by 18
- 2024PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wildcited by 17
- 2023Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessmentcited by 16
- 2021StarVQA: Space-Time Attention for Video Quality Assessmentcited by 15
- 2021A strong baseline for image and video quality assessmentcited by 15
- 2024VQA2: Visual Question Answering for Video Quality Assessmentcited by 15
- 2024Video Quality Assessment: A Comprehensive Surveycited by 15
- 2022Subjective and Objective Analysis of Streamed Gaming Videoscited by 14
- 2023Perceptual Quality Assessment of Face Video Compression: A Benchmark and An Effective Methodcited by 14
- 2025NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: Methods and Resultscited by 14
- 2024Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessmentcited by 13
- 2024Exploring AIGC Video Quality: A Focus on Visual Harmony, Video-Text Consistency and Domain Distribution Gapcited by 13
- 2024Benchmarking Multi-dimensional AIGC Video Quality Assessment: A Dataset and Unified Modelcited by 13
- 2025FVQ: A Large-Scale Dataset and an LMM-based Method for Face Video Quality Assessmentcited by 13
- 2023Light-VQA: A Multi-Dimensional Quality Assessment Model for Low-Light Video Enhancementcited by 12
- 2023Vulnerabilities in Video Quality Assessment Models: The Challenge of Adversarial Attackscited by 12
- 2023CLiF-VQA: Enhancing Video Quality Assessment by Incorporating High-Level Semantic Information related to Human Feelingscited by 11
- 2024CLIPVQA:Video Quality Assessment via CLIPcited by 11
50 most cited of 147 All 147 (JSON)
- 2026High-Fidelity Video Quality Assessment with VQA-Specific Saliency15 Sep 2026
- 2026Bridging the Perceptual Gap: Residual-Enhanced Downscaling and Manifold-Aware Perception Alignment Adaptation for NR-IQA15 Sep 2026
- 2026Neural video codecs quality assessment dataset and benchmark29 Aug 2026
- 2026CamWorldQA: Perceptual Quality Assessment of Camera-Controlled World Video Generation19 Aug 2026
- 2026LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment11 Aug 2026
- 2026TREND-10K: A Comprehensive Dataset for Next-Generation Video Quality Assessment Based on Preference-Driven Media11 Aug 2026
- 2026Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model18 Jul 2026
- 2026Contrastive Order Learning: A General Framework for Ordinal Regression9 Jul 2026
- 2026VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment20 May 2026
- 2026FGSVQA: Frequency-Guided Short-form Video Quality Assessment19 May 2026
- 2026Unleashing Vision Transformer Potential In Image Quality Assessment via Global-Local Adaptive Interaction18 May 2026
- 2026GameScope: A Multi-Attribute, Multi-Codec Benchmark Dataset for Gaming Video Quality Assessment2 May 2026
38 more
- 2026Subjective Portrait Region Cropping in Landscape Videos with Temporal Annotation Smoothing27 Apr 2026
- 2026DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment14 Apr 2026
- 2026Pioneering Perceptual Video Fluency Assessment: A Novel Task with Benchmark Dataset and Baseline27 Mar 2026
- 2026VQ-Jarvis: Retrieval-Augmented Video Restoration Agent with Sharp Vision and Fast Thought24 Mar 2026
- 2026MDS-VQA: Model-Informed Data Selection for Video Quality Assessment12 Mar 2026
- 2026Seeing Beyond 8bits: Subjective and Objective Quality Assessment of HDR-UGC Videos1 Mar 2026
- 2026Scaling Audio-Visual Quality Assessment Dataset via Crowdsourcing26 Feb 2026
- 2026Analytic Score Optimization for Multi Dimension Video Quality Assessment18 Feb 2026
- 2026VideoAesBench: Benchmarking the Video Aesthetics Perception Capabilities of Large Multimodal Models29 Jan 2026
- 2026QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding26 Jan 2026
- 2026Convolutions Need Registers Too: HVS-Inspired Dynamic Attention for Video Quality Assessment16 Jan 2026
- 2025VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment22 Nov 2025
- 2025CAMP-VQA: Caption-Embedded Multimodal Perception for No-Reference Quality Assessment of Compressed Video10 Nov 2025
- 2025PreResQ-R1: Response-Preference Disentangled Ranking-and-Scoring Reinforcement Optimization for Robust Visual Quality Assessment7 Nov 2025
- 2025NovisVQ: A Streaming Convolutional Neural Network for No-Reference Opinion-Unaware Frame Quality Assessment6 Nov 2025
- 2025VADB: A Large-Scale Video Aesthetic Database with Professional and Multi-Dimensional Annotations29 Oct 2025
- 2025No-Reference Rendered Video Quality Assessment: Dataset and Metrics15 Oct 2025
- 2025DREAM: A Benchmark Study for Deepfake photoREalism AssessMent11 Oct 2025
- 2025CHUG: Crowdsourced User-Generated HDR Video Quality Dataset10 Oct 2025
- 2025Q-Router: Agentic Video Quality Assessment with Expert Model Routing and Artifact Localization9 Oct 2025
- 2025MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessment15 Sep 2025
- 2025VQualA 2025 Challenge on Engagement Prediction for Short Videos: Methods and Results3 Sep 2025
- 2025Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding28 Aug 2025
- 2025DIVA-VQA: Detecting Inter-frame Variations in UGC Video Quality14 Aug 2025
- 2025Frequency-Assisted Adaptive Sharpening Scheme Considering Bitrate and Quality Tradeoff12 Aug 2025
- 2025Adaptive High-Frequency Preprocessing for Video Coding12 Aug 2025
- 2025Subjective and Objective Quality Assessment of Banding Artifacts on Compressed Videos12 Aug 2025
- 2025Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal Adaptation8 Aug 2025
- 2025VQAThinker: Exploring Generalizable and Explainable Video Quality Assessment via Reinforcement Learning8 Aug 2025
- 2025Refine-IQA: Multi-Stage Reinforcement Finetuning for Perceptual Image Quality Assessment4 Aug 2025
- 2025Engagement Prediction of Short Videos with Large Multimodal Models4 Aug 2025
- 2025LEHA-CVQAD: Dataset To Enable Generalized Video Quality Assessment of Compression Artifacts5 Jul 2025
- 2025RGC-VQA: An Exploration Database for Robotic-Generated Video Quality Assessment30 Jun 2025
- 2025Score2Instruct: Scaling Up Video Quality-Centric Instructions via Automated Dimension Scoring26 Jun 2025
- 2025VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learning23 Jun 2025
- 2025Scaling-up Perceptual Video Quality Assessment28 May 2025
- 2025Context and Pixel Aware Large Language Model for Video Quality Assessment21 May 2025
- 2025Generalizable Video Quality Assessment via Weak-to-Strong Learning6 May 2025
50 newest of 147 All 147 (JSON)
- 2024Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMscited by 39
- 2024LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Modelscited by 28
- 2024VQA2: Visual Question Answering for Video Quality Assessmentcited by 15
- 2025Scaling-up Perceptual Video Quality Assessmentcited by 5
- 2024An Ensemble Approach to Short-form Video Quality Assessment Using Multimodal LLMcited by 4
- 2025Engagement Prediction of Short Videos with Large Multimodal Modelscited by 4
- 2025Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal Adaptationcited by 3
- 2025DVLTA-VQA: Decoupled Vision-Language Modeling with Text-Guided Adaptation for Blind Video Quality Assessmentcited by 1
- 2025Context and Pixel Aware Large Language Model for Video Quality Assessmentcited by 1
- 2025VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessmentcited by 1
- 2026VideoAesBench: Benchmarking the Video Aesthetics Perception Capabilities of Large Multimodal Modelscited by 1
- 2026DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessmentcited by 1
12 most cited of 14 All 14 (JSON)
- 2026Bridging the Perceptual Gap: Residual-Enhanced Downscaling and Manifold-Aware Perception Alignment Adaptation for NR-IQA15 Sep 2026
- 2026VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment20 May 2026
- 2026DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment14 Apr 2026
- 2026VideoAesBench: Benchmarking the Video Aesthetics Perception Capabilities of Large Multimodal Models29 Jan 2026
- 2025VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment22 Nov 2025
- 2025Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal Adaptation8 Aug 2025
- 2025Engagement Prediction of Short Videos with Large Multimodal Models4 Aug 2025
- 2025Scaling-up Perceptual Video Quality Assessment28 May 2025
- 2025Context and Pixel Aware Large Language Model for Video Quality Assessment21 May 2025
- 2025DVLTA-VQA: Decoupled Vision-Language Modeling with Text-Guided Adaptation for Blind Video Quality Assessment16 Apr 2025
- 2024An Ensemble Approach to Short-form Video Quality Assessment Using Multimodal LLM24 Dec 2024
- 2024VQA2: Visual Question Answering for Video Quality Assessment6 Nov 2024
12 newest of 14 All 14 (JSON)
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
- 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
- 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
- 2022A Deep Learning based No-reference Quality Assessment Model for UGC Videoscited by 128
- 2021RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Contentcited by 100
- 2021Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perceptioncited by 97
- 2022Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessmentcited by 50
- 2024Towards A Better Metric for Text-to-Video Generationcited by 50
- 2023MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videoscited by 46
- 2023Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approachcited by 45
- 2022DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessmentcited by 41
12 most cited of 124 All 124 (JSON)
- 2026High-Fidelity Video Quality Assessment with VQA-Specific Saliency15 Sep 2026
- 2026Neural video codecs quality assessment dataset and benchmark29 Aug 2026
- 2026CamWorldQA: Perceptual Quality Assessment of Camera-Controlled World Video Generation19 Aug 2026
- 2026LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment11 Aug 2026
- 2026TREND-10K: A Comprehensive Dataset for Next-Generation Video Quality Assessment Based on Preference-Driven Media11 Aug 2026
- 2026Contrastive Order Learning: A General Framework for Ordinal Regression9 Jul 2026
- 2026FGSVQA: Frequency-Guided Short-form Video Quality Assessment19 May 2026
- 2026Unleashing Vision Transformer Potential In Image Quality Assessment via Global-Local Adaptive Interaction18 May 2026
- 2026GameScope: A Multi-Attribute, Multi-Codec Benchmark Dataset for Gaming Video Quality Assessment2 May 2026
- 2026Subjective Portrait Region Cropping in Landscape Videos with Temporal Annotation Smoothing27 Apr 2026
- 2026Pioneering Perceptual Video Fluency Assessment: A Novel Task with Benchmark Dataset and Baseline27 Mar 2026
- 2026MDS-VQA: Model-Informed Data Selection for Video Quality Assessment12 Mar 2026
12 newest of 124 All 124 (JSON)