# Investigate the Low-level Visual Perception in Vision-Language based Image Quality Assessment

arXiv: https://arxiv.org/abs/2512.09573 · HTML: https://mlsanity.com/paper/2512.09573/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 16 of its 19 references that are in the map.
- First posted on arXiv 10 Dec 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 16 of 16)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [Massive Online Crowdsourced Study of Subjective and Objective Picture Quality](https://mlsanity.com/paper/1511.02919/) (2015)
- [mPLUG-2: A Modularized Multi-modal Foundation Model Across Text, Image and Video](https://mlsanity.com/paper/2302.00402/) (2023)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models](https://mlsanity.com/paper/2312.08962/) (2023)
- [Q-Ground: Image Quality Grounding with Large Multi-modality Models](https://mlsanity.com/paper/2407.17035/) (2024)
- [SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning](https://mlsanity.com/paper/2411.10161/) (2024)

## Built on by (most cited first, 3 of 3)

- [IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment](https://mlsanity.com/paper/2607.12375/) (2026)
- [VQ-Jarvis: Retrieval-Augmented Video Restoration Agent with Sharp Vision and Fast Thought](https://mlsanity.com/paper/2603.22998/) (2026)
- [Geometric Similarity in VLM Low-Level Vision Representations](https://mlsanity.com/paper/2610.00848/) (2026)

## Abstract (arXiv)

Recent advances in Image Quality Assessment (IQA) have leveraged Multi-modal Large Language Models (MLLMs) to generate descriptive explanations. However, despite their strong visual perception modules, these models often fail to reliably detect basic low-level distortions such as blur, noise, and compression, and may produce inconsistent evaluations across repeated inferences. This raises an essential question: do MLLM-based IQA systems truly perceive the visual features that matter? To examine this issue, we introduce a low-level distortion perception task that requires models to classify specific distortion types. Our component-wise analysis shows that although MLLMs are structurally capable of representing such distortions, they tend to overfit training templates, leading to biases in quality scoring. As a result, critical low-level features are weakened or lost during the vision-language alignment transfer stage. Furthermore, by computing the semantic distance between visual features and corresponding semantic tokens before and after component-wise fine-tuning, we show that improving the alignment of the vision encoder dramatically enhances distortion recognition accuracy, increasing it from 14.92% to 84.43%. Overall, these findings indicate that incorporating dedicated constraints on the vision encoder can strengthen text-explainable visual representations and enable MLLM-based pipelines to produce more coherent and interpretable reasoning in vision-centric tasks.