# AIM 2024 Challenge on Compressed Video Quality Assessment: Methods and Results

arXiv: https://arxiv.org/abs/2408.11982 · HTML: https://mlsanity.com/paper/2408.11982/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 58 references that are in the map.
- First posted on arXiv 21 Aug 2024.

## Builds on (most cited first, 29 of 29)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling](https://mlsanity.com/paper/2207.02595/) (2022)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [RankIQA: Learning from Rankings for No-reference Image Quality Assessment](https://mlsanity.com/paper/1707.08347/) (2017)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [D3D: Distilled 3D Networks for Video Action Recognition](https://mlsanity.com/paper/1812.08249/) (2018)
- [A Haar Wavelet-Based Perceptual Similarity Index for Image Quality Assessment](https://mlsanity.com/paper/1607.06140/) (2016)
- [Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network](https://mlsanity.com/paper/2204.10485/) (2022)
- [Norm-in-Norm Loss with Faster Convergence and Better Performance for Image Quality Assessment](https://mlsanity.com/paper/2008.03889/) (2020)
- [PyTorch Image Quality: Metrics for Image Quality Assessment](https://mlsanity.com/paper/2208.14818/) (2022)
- [VideoSet: A Large-Scale Compressed Video Quality Dataset Based on JND Measurement](https://mlsanity.com/paper/1701.01500/) (2017)
- [Video compression dataset and benchmark of learning-based video-quality metrics](https://mlsanity.com/paper/2211.12109/) (2022)

## Built on by (most cited first, 7 of 7)

- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)
- [AIM 2024 Challenge on Video Saliency Prediction: Methods and Results](https://mlsanity.com/paper/2409.14827/) (2024)
- [AIM 2024 Challenge on Efficient Video Super-Resolution for AV1 Compressed Content](https://mlsanity.com/paper/2409.17256/) (2024)
- [AIM 2024 Challenge on UHD Blind Photo Quality Assessment](https://mlsanity.com/paper/2409.16271/) (2024)
- [AIM 2024 Sparse Neural Rendering Challenge: Methods and Results](https://mlsanity.com/paper/2409.15045/) (2024)
- [Compressed Depth Map Super-Resolution and Restoration: AIM 2024 Challenge Results](https://mlsanity.com/paper/2409.16277/) (2024)
- [AIM 2024 Challenge on Video Super-Resolution Quality Assessment: Methods and Results](https://mlsanity.com/paper/2410.04225/) (2024)

## Abstract (arXiv)

Video quality assessment (VQA) is a crucial task in the development of video compression standards, as it directly impacts the viewer experience. This paper presents the results of the Compressed Video Quality Assessment challenge, held in conjunction with the Advances in Image Manipulation (AIM) workshop at ECCV 2024. The challenge aimed to evaluate the performance of VQA methods on a diverse dataset of 459 videos, encoded with 14 codecs of various compression standards (AVC/H.264, HEVC/H.265, AV1, and VVC/H.266) and containing a comprehensive collection of compression artifacts. To measure the methods performance, we employed traditional correlation coefficients between their predictions and subjective scores, which were collected via large-scale crowdsourced pairwise human comparisons. For training purposes, participants were provided with the Compressed Video Quality Assessment Dataset (CVQAD), a previously developed dataset of 1022 videos. Up to 30 participating teams registered for the challenge, while we report the results of 6 teams, which submitted valid final solutions and code for reproducing the results. Moreover, we calculated and present the performance of state-of-the-art VQA methods on the developed dataset, providing a comprehensive benchmark for future research. The dataset, results, and online leaderboard are publicly available at https://challenges.videoprocessing.ai/challenges/compressedvideo-quality-assessment.html.