# Evaluating Vision-Language Models for Image Quality Assessment using Psychophysical Data

arXiv: https://arxiv.org/abs/2603.24578 · HTML: https://mlsanity.com/paper/2603.24578/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 3 of its 26 references that are in the map.
- First posted on arXiv 25 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 3 of 3)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Perceptual Quality Prediction on Authentically Distorted Images Using a Bag of Features Approach](https://mlsanity.com/paper/1609.04757/) (2016)
- [Perceptual Tone Mapping Model for High Dynamic Range Imaging](https://mlsanity.com/paper/2309.16975/) (2023)

## Abstract (arXiv)

Psychophysical experiments remain the most reliable approach for perceptual image quality assessment (IQA), yet their cost and limited scalability motivate automated alternatives. This paper investigates whether Vision-Language Models (VLMs) can assist in assessing perceived image appearance and quality. We introduce a psychophysics-inspired framework to probe VLM perceptual sensitivity through controlled pairwise image comparisons of contrast, colorfulness, and overall preference. Six VLMs (four proprietary and two open-weight models) are compared against psychophysical data. Results reveal strong attribute-dependent variability: Claude exhibits the highest internal consistency, whereas GPT achieves the strongest agreement for overall preference. Claude and Qwen show the strongest alignment for colorfulness, while Qwen performs best for contrast. However, no model consistently matches human perception across all attributes. High self-consistency does not necessarily imply perceptual validity, and VLM--human agreement generally improves when perceptual differences among renderings are more pronounced.