# Unified Quality Assessment of In-the-Wild Videos with Mixed Datasets Training

arXiv: https://arxiv.org/abs/2011.04263 · HTML: https://mlsanity.com/paper/2011.04263/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 30 of 72)

- [EvalCrafter: Benchmarking and Evaluating Large Video Generation Models](https://mlsanity.com/paper/2310.11440/) (2023)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling](https://mlsanity.com/paper/2207.02595/) (2022)
- [A Deep Learning based No-reference Quality Assessment Model for UGC Videos](https://mlsanity.com/paper/2204.14047/) (2022)
- [RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Content](https://mlsanity.com/paper/2101.10955/) (2021)
- [Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perception](https://mlsanity.com/paper/2108.08505/) (2021)
- [Towards A Better Metric for Text-to-Video Generation](https://mlsanity.com/paper/2401.07781/) (2024)
- [Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessment](https://mlsanity.com/paper/2210.05357/) (2022)
- [Towards Explainable In-the-Wild Video Quality Assessment: A Database and a Language-Prompted Approach](https://mlsanity.com/paper/2305.12726/) (2023)
- [Perceptual Video Quality Assessment: A Survey](https://mlsanity.com/paper/2402.03413/) (2024)
- [DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessment](https://mlsanity.com/paper/2206.09853/) (2022)
- [KVQ: Kwai Video Quality Assessment for Short-form Videos](https://mlsanity.com/paper/2402.07220/) (2024)
- [PTab: Using the Pre-trained Language Model for Modeling Tabular Data](https://mlsanity.com/paper/2209.08060/) (2022)
- [Video compression dataset and benchmark of learning-based video-quality metrics](https://mlsanity.com/paper/2211.12109/) (2022)
- [Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterion](https://mlsanity.com/paper/2302.13269/) (2023)
- [ChipQA: No-Reference Video Quality Prediction via Space-Time Chips](https://mlsanity.com/paper/2109.08726/) (2021)
- [Zoom-VQA: Patches, Frames and Clips Integration for Video Quality Assessment](https://mlsanity.com/paper/2304.06440/) (2023)
- [Ada-DQA: Adaptive Diverse Quality-aware Feature Acquisition for Video Quality Assessment](https://mlsanity.com/paper/2308.00729/) (2023)
- [CONVIQT: Contrastive Video Quality Estimator](https://mlsanity.com/paper/2206.14713/) (2022)
- [Capturing Co-existing Distortions in User-Generated Content for No-reference Video Quality Assessment](https://mlsanity.com/paper/2307.16813/) (2023)
- [Universal Perturbation Attack on Differentiable No-Reference Image- and Video-Quality Metrics](https://mlsanity.com/paper/2211.00366/) (2022)
- [PTM-VQA: Efficient Video Quality Assessment Leveraging Diverse PreTrained Models from the Wild](https://mlsanity.com/paper/2405.17765/) (2024)
- [No-Reference Image Quality Assessment by Hallucinating Pristine Features](https://mlsanity.com/paper/2108.04165/) (2021)
- [Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessment](https://mlsanity.com/paper/2304.14672/) (2023)
- [StarVQA: Space-Time Attention for Video Quality Assessment](https://mlsanity.com/paper/2108.09635/) (2021)
- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)
- [Comparing the Robustness of Modern No-Reference Image- and Video-Quality Metrics to Adversarial Attacks](https://mlsanity.com/paper/2310.06958/) (2023)
- [PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Prompts](https://mlsanity.com/paper/2403.04993/) (2024)
- [VisualCritic: Making LMMs Perceive Visual Quality Like Humans](https://mlsanity.com/paper/2403.12806/) (2024)
- [Vulnerabilities in Video Quality Assessment Models: The Challenge of Adversarial Attacks](https://mlsanity.com/paper/2309.13609/) (2023)

## Abstract (arXiv)

Video quality assessment (VQA) is an important problem in computer vision. The videos in computer vision applications are usually captured in the wild. We focus on automatically assessing the quality of in-the-wild videos, which is a challenging problem due to the absence of reference videos, the complexity of distortions, and the diversity of video contents. Moreover, the video contents and distortions among existing datasets are quite different, which leads to poor performance of data-driven methods in the cross-dataset evaluation setting. To improve the performance of quality assessment models, we borrow intuitions from human perception, specifically, content dependency and temporal-memory effects of human visual system. To face the cross-dataset evaluation challenge, we explore a mixed datasets training strategy for training a single VQA model with multiple datasets. The proposed unified framework explicitly includes three stages: relative quality assessor, nonlinear mapping, and dataset-specific perceptual scale alignment, to jointly predict relative quality, perceptual quality, and subjective quality. Experiments are conducted on four publicly available datasets for VQA in the wild, i.e., LIVE-VQC, LIVE-Qualcomm, KoNViD-1k, and CVD2014. The experimental results verify the effectiveness of the mixed datasets training strategy and prove the superior performance of the unified model in comparison with the state-of-the-art models. For reproducible research, we make the PyTorch implementation of our method available at https://github.com/lidq92/MDTVSFA.