# UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment

arXiv: https://arxiv.org/abs/2406.01069 · HTML: https://mlsanity.com/paper/2406.01069/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 58 of its 120 references that are in the map.
- First posted on arXiv 3 Jun 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 30 of 58)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [PaLM-E: An Embodied Multimodal Language Model](https://mlsanity.com/paper/2303.03378/) (2023)
- [CoCa: Contrastive Captioners are Image-Text Foundation Models](https://mlsanity.com/paper/2205.01917/) (2022)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [EVA-CLIP: Improved Training Techniques for CLIP at Scale](https://mlsanity.com/paper/2303.15389/) (2023)
- [FILIP: Fine-grained Interactive Language-Image Pre-Training](https://mlsanity.com/paper/2111.07783/) (2021)
- [Otter: A Multi-Modal Model with In-Context Instruction Tuning](https://mlsanity.com/paper/2305.03726/) (2023)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [MaxViT: Multi-Axis Vision Transformer](https://mlsanity.com/paper/2204.01697/) (2022)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessment](https://mlsanity.com/paper/1910.06180/) (2019)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [SPHINX: The Joint Mixing of Weights, Tasks, and Visual Embeddings for Multi-modal Large Language Models](https://mlsanity.com/paper/2311.07575/) (2023)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023)
- [GPT4Tools: Teaching Large Language Model to Use Tools via Self-instruction](https://mlsanity.com/paper/2305.18752/) (2023)
- [Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessment](https://mlsanity.com/paper/1612.01697/) (2016)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [Massive Online Crowdsourced Study of Subjective and Objective Picture Quality](https://mlsanity.com/paper/1511.02919/) (2015)
- [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021)
- [MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2004.05508/) (2020)

## Built on by (most cited first, 13 of 13)

- [Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Model](https://mlsanity.com/paper/2503.06141/) (2025)
- [NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)](https://mlsanity.com/paper/2604.12512/) (2026)
- [HaploVL: A Single-Transformer Baseline for Multi-Modal Understanding](https://mlsanity.com/paper/2503.14694/) (2025)
- [Gamma: Toward Generic Image Assessment with Mixture of Assessment Experts](https://mlsanity.com/paper/2503.06678/) (2025)
- [LoRA-Gen: Specializing Large Language Model via Online LoRA Generation](https://mlsanity.com/paper/2506.11638/) (2025)
- [InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animation](https://mlsanity.com/paper/2504.10905/) (2025)
- [Visual Aesthetic Benchmark: Can Frontier Models Judge Beauty?](https://mlsanity.com/paper/2605.12684/) (2026)
- [Remote Sensing Semantic Segmentation Quality Assessment based on Vision Language Model](https://mlsanity.com/paper/2502.13990/) (2025)
- [Exploiting Discriminative Codebook Prior for Autoregressive Image Generation](https://mlsanity.com/paper/2508.10719/) (2025)
- [Q-Hawkeye: Reliable Visual Policy Optimization for Image Quality Assessment](https://mlsanity.com/paper/2601.22920/) (2026)
- [What Do Vision-Language Models Encode for Personalized Image Aesthetics Assessment?](https://mlsanity.com/paper/2604.11374/) (2026)
- [Aesthetic Image Captioning with Saliency Enhanced MLLMs](https://mlsanity.com/paper/2509.04378/) (2025)
- [MultiEmo-Bench: Multi-label Visual Emotion Analysis for Multi-modal Large Language Models](https://mlsanity.com/paper/2605.14635/) (2026)

## Abstract (arXiv)

Image Quality Assessment (IQA) and Image Aesthetic Assessment (IAA) aim to simulate human subjective perception of image visual quality and aesthetic appeal. Despite distinct learning objectives, they have underlying interconnectedness due to consistent human assessment perception. In this paper, we propose Unified vision-language pre-training of Quality and Aesthetics (UniQA}), to extract useful and common representations from two tasks, thereby benefiting them simultaneously. However, the lack of text in the IQA datasets and the textual noise in the IAA datasets pose severe challenges for multimodal pre-training. To address this, we (1) utilize multimodal large language models (MLLMs) to generate high-quality text descriptions; (2) use the generated text for IAA as metadata to purify noisy IAA data. To effectively adapt the pre-trained UniQA to downstream tasks, we further propose a lightweight adapter that utilizes versatile cues to fully exploit the extensive knowledge of the pre-trained model. UniQA demonstrates high competitiveness in various image assessment tasks, including classical IQA and IAA tasks, few-label IQA, and other downstream tasks, showing promise as a foundational assessment model. Codes are available at https://github.com/zht8506/UniQA.