# Q-CLIP: Unleashing the Power of Vision-Language Models for Video Quality Assessment through Unified Cross-Modal Adaptation

arXiv: https://arxiv.org/abs/2508.06092 · HTML: https://mlsanity.com/paper/2508.06092/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 34 of its 53 references that are in the map.
- First posted on arXiv 8 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Features](https://mlsanity.com/paper/2502.14786/) (2025)
- [CLIP-Adapter: Better Vision-Language Models with Feature Adapters](https://mlsanity.com/paper/2110.04544/) (2021)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Vision-Language Models for Vision Tasks: A Survey](https://mlsanity.com/paper/2304.00685/) (2023)
- [VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understanding](https://mlsanity.com/paper/2109.14084/) (2021)
- [Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning](https://mlsanity.com/paper/2203.02053/) (2022)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Tip-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling](https://mlsanity.com/paper/2111.03930/) (2021)
- [Expanding Language-Image Pretrained Models for General Video Recognition](https://mlsanity.com/paper/2208.02816/) (2022)
- [Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectives](https://mlsanity.com/paper/2211.04894/) (2022)
- [Prompting Visual-Language Models for Efficient Video Understanding](https://mlsanity.com/paper/2112.04478/) (2021)
- [FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Sampling](https://mlsanity.com/paper/2207.02595/) (2022)
- [Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognition](https://mlsanity.com/paper/1708.07632/) (2017)
- [Patch-VQ: 'Patching Up' the Video Quality Problem](https://mlsanity.com/paper/2011.13544/) (2020)
- [Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution](https://mlsanity.com/paper/2501.11561/) (2025)
- [UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Content](https://mlsanity.com/paper/2005.14354/) (2020)
- [A Deep Learning based No-reference Quality Assessment Model for UGC Videos](https://mlsanity.com/paper/2204.14047/) (2022)
- [Large-Scale Study of Perceptual Video Quality](https://mlsanity.com/paper/1803.01761/) (2018)
- [YouTube UGC Dataset for Video Compression Research](https://mlsanity.com/paper/1904.06457/) (2019)
- [Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perception](https://mlsanity.com/paper/2108.08505/) (2021)
- [Diagnosing and Rectifying Vision Models using Language](https://mlsanity.com/paper/2302.04269/) (2023)
- [Learning Generalized Spatial-Temporal Deep Feature Representation for No-Reference Video Quality Assessment](https://mlsanity.com/paper/2012.13936/) (2020)
- [MD-VQA: Multi-Dimensional Quality Assessment for UGC Live Videos](https://mlsanity.com/paper/2303.14933/) (2023)

## Built on by (most cited first, 3 of 3)

- [Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment](https://mlsanity.com/paper/2607.13941/) (2026)
- [A Lightweight Multimodal Vision-Language Framework for Early-Stage Anatomical Green Fruit Classification in Commercial Orchards](https://mlsanity.com/paper/2608.24935/) (2026)
- [LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment](https://mlsanity.com/paper/2609.26205/) (2026)

## Abstract (arXiv)

Accurate and efficient Video Quality Assessment (VQA) has long been a key research challenge. Current mainstream VQA methods typically improve performance by pretraining on large-scale classification datasets (e.g., ImageNet, Kinetics-400), followed by fine-tuning on VQA datasets. However, this strategy presents two significant challenges: (1) merely transferring semantic knowledge learned from pretraining is insufficient for VQA, as video quality depends on multiple factors (e.g., semantics, distortion, motion, aesthetics); (2) pretraining on large-scale datasets demands enormous computational resources, often dozens or even hundreds of times greater than training directly on VQA datasets. Recently, Vision-Language Models (VLMs) have shown remarkable generalization capabilities across a wide range of visual tasks, and have begun to demonstrate promising potential in quality assessment. In this work, we propose Q-CLIP, the first fully VLMs-based framework for VQA. Q-CLIP enhances both visual and textual representations through a Shared Cross-Modal Adapter (SCMA), which contains only a minimal number of trainable parameters and is the only component that requires training. This design significantly reduces computational cost. In addition, we introduce a set of five learnable quality-level prompts to guide the VLMs in perceiving subtle quality variations, thereby further enhancing the model's sensitivity to video quality. Furthermore, we investigate the impact of different frame sampling strategies on VQA performance, and find that frame-difference-based sampling leads to better generalization performance across datasets. Extensive experiments demonstrate that Q-CLIP exhibits excellent performance on several VQA datasets.