# Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment

arXiv: https://arxiv.org/abs/2401.02614 · HTML: https://mlsanity.com/paper/2401.02614/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 5 Jan 2024.

## Builds on (most cited first, 23 of 23)

- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling](https://mlsanity.com/paper/2207.02595/) (2022)
- [Quality Assessment of In-the-Wild Videos](https://mlsanity.com/paper/1908.00375/) (2019)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perception](https://mlsanity.com/paper/2108.08505/) (2021)
- [Re-IQA: Unsupervised Learning for Image Quality Assessment in the Wild](https://mlsanity.com/paper/2304.00451/) (2023)
- [Image Quality Assessment using Contrastive Learning](https://mlsanity.com/paper/2110.13266/) (2021)
- [Data-Efficient Image Quality Assessment with Attention-Panel Decoder](https://mlsanity.com/paper/2304.04952/) (2023)
- [MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos](https://mlsanity.com/paper/2303.14933/) (2023)
- [Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterion](https://mlsanity.com/paper/2302.13269/) (2023)

## Built on by (most cited first, 13 of 13)

- [NTIRE 2024 Quality Assessment of AI-Generated Content Challenge](https://mlsanity.com/paper/2404.16687/) (2024)
- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)
- [ReLaX-VQA: Residual Fragment and Layer Stack Extraction for Enhancing Video Quality Assessment](https://mlsanity.com/paper/2407.11496/) (2024)
- [Video Quality Assessment for Online Processing: From Spatial to Temporal Sampling](https://mlsanity.com/paper/2501.07087/) (2025)
- [MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessment](https://mlsanity.com/paper/2504.16003/) (2025)
- [DIVA-VQA: Detecting Inter-frame Variations in UGC Video Quality](https://mlsanity.com/paper/2508.10605/) (2025)
- [CAMP-VQA: Caption-Embedded Multimodal Perception for No-Reference Quality Assessment of Compressed Video](https://mlsanity.com/paper/2511.07290/) (2025)
- [Multi-Dimensional Quality Assessment for AI-Generated Human-Centric Videos: Dataset and Model](https://mlsanity.com/paper/2607.16742/) (2026)
- [Advancing Video Quality Assessment for AIGC](https://mlsanity.com/paper/2409.14888/) (2024)
- [DiffVQA: Video Quality Assessment Using Diffusion Feature Extractor](https://mlsanity.com/paper/2505.03261/) (2025)
- [EduVQA: Towards Concept-Aware Assessment of Educational AI-Generated Videos](https://mlsanity.com/paper/2603.03066/) (2026)
- [Subjective and Objective Quality Assessment of Banding Artifacts on Compressed Videos](https://mlsanity.com/paper/2508.08700/) (2025)
- [Bridging the Perceptual Gap: Residual-Enhanced Downscaling and Manifold-Aware Perception Alignment Adaptation for NR-IQA](https://mlsanity.com/paper/2609.16664/) (2026)

## Abstract (arXiv)

Quality assessment of images and videos emphasizes both local details and global semantics, whereas general data sampling methods (e.g., resizing, cropping or grid-based fragment) fail to catch them simultaneously. To address the deficiency, current approaches have to adopt multi-branch models and take as input the multi-resolution data, which burdens the model complexity. In this work, instead of stacking up models, a more elegant data sampling method (named as SAMA, scaling and masking) is explored, which compacts both the local and global content in a regular input size. The basic idea is to scale the data into a pyramid first, and reduce the pyramid into a regular data dimension with a masking strategy. Benefiting from the spatial and temporal redundancy in images and videos, the processed data maintains the multi-scale characteristics with a regular input size, thus can be processed by a single-branch model. We verify the sampling method in image and video quality assessment. Experiments show that our sampling method can improve the performance of current single-branch models significantly, and achieves competitive performance to the multi-branch models without extra model complexity. The source code will be available at https://github.com/Sissuire/SAMA.