# A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment

arXiv: https://arxiv.org/abs/2403.10854 · HTML: https://mlsanity.com/paper/2403.10854/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 55 of its 77 references that are in the map.
- First posted on arXiv 16 Mar 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Prefix-Tuning: Optimizing Continuous Prompts for Generation](https://mlsanity.com/paper/2101.00190/) (2021)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond](https://mlsanity.com/paper/2308.12966/) (2023)
- [Kosmos-2: Grounding Multimodal Large Language Models to the World](https://mlsanity.com/paper/2306.14824/) (2023)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)](https://mlsanity.com/paper/2309.17421/) (2023)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Model](https://mlsanity.com/paper/2304.15010/) (2023)
- [mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaboration](https://mlsanity.com/paper/2311.04257/) (2023)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)

## Built on by (most cited first, 30 of 43)

- [Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution](https://mlsanity.com/paper/2501.11561/) (2025)
- [VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rank](https://mlsanity.com/paper/2505.14460/) (2025)
- [Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare](https://mlsanity.com/paper/2405.19298/) (2024)
- [T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation](https://mlsanity.com/paper/2501.12612/) (2025)
- [Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMs](https://mlsanity.com/paper/2409.20063/) (2024)
- [Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Dataset](https://mlsanity.com/paper/2405.18842/) (2024)
- [FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models](https://mlsanity.com/paper/2404.13306/) (2024)
- [AutoVFX: Physically Realistic Video Editing from Natural Language Instructions](https://mlsanity.com/paper/2411.02394/) (2024)
- [Visual Text Processing: A Comprehensive Review and Unified Evaluation](https://mlsanity.com/paper/2504.21682/) (2025)
- [The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspective](https://mlsanity.com/paper/2407.08583/) (2024)
- [HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluator](https://mlsanity.com/paper/2411.17261/) (2024)
- [Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessment](https://mlsanity.com/paper/2411.17237/) (2024)
- [Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Model](https://mlsanity.com/paper/2503.06141/) (2025)
- [Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Ranking](https://mlsanity.com/paper/2509.25787/) (2025)
- [From Text to Pixel: Advancing Long-Context Understanding in MLLMs](https://mlsanity.com/paper/2405.14213/) (2024)
- [Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessment](https://mlsanity.com/paper/2410.02505/) (2024)
- [MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMs](https://mlsanity.com/paper/2510.01691/) (2025)
- [PocketSR: The Super-Resolution Expert in Your Pocket Mobiles](https://mlsanity.com/paper/2510.03012/) (2025)
- [Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuning](https://mlsanity.com/paper/2504.01655/) (2025)
- [REVEAL: Reasoning-Enhanced Forensic Evidence Analysis for Explainable AI-Generated Image Detection](https://mlsanity.com/paper/2511.23158/) (2025)
- [Modification Takes Courage: Seamless Image Stitching via Reference-Driven Inpainting](https://mlsanity.com/paper/2411.10309/) (2024)
- [An Ensemble Approach to Short-form Video Quality Assessment Using Multimodal LLM](https://mlsanity.com/paper/2412.18060/) (2024)
- [Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunities](https://mlsanity.com/paper/2410.08534/) (2024)
- [Enhancing Explainability in Multimodal Large Language Models Using Ontological Context](https://mlsanity.com/paper/2409.18753/) (2024)
- [DGIQA: Depth-guided Feature Attention and Refinement for Generalizable Image Quality Assessment](https://mlsanity.com/paper/2505.24002/) (2025)
- [DeQA-Doc: Adapting DeQA-Score to Document Image Quality Assessment](https://mlsanity.com/paper/2507.12796/) (2025)
- [Creative4U: MLLMs-based Advertising Creative Image Selector with Comparative Reasoning](https://mlsanity.com/paper/2508.12628/) (2025)
- [Enhancing Diffusion-based Restoration Models via Difficulty-Adaptive Reinforcement Learning with IQA Reward](https://mlsanity.com/paper/2511.01645/) (2025)
- [QualiRAG: Retrieval-Augmented Generation for Visual Quality Understanding](https://mlsanity.com/paper/2601.18195/) (2026)
- [Context and Pixel Aware Large Language Model for Video Quality Assessment](https://mlsanity.com/paper/2505.16025/) (2025)

## Abstract (arXiv)

While Multimodal Large Language Models (MLLMs) have experienced significant advancement in visual understanding and reasoning, their potential to serve as powerful, flexible, interpretable, and text-driven models for Image Quality Assessment (IQA) remains largely unexplored. In this paper, we conduct a comprehensive and systematic study of prompting MLLMs for IQA. We first investigate nine prompting systems for MLLMs as the combinations of three standardized testing procedures in psychophysics (i.e., the single-stimulus, double-stimulus, and multiple-stimulus methods) and three popular prompting strategies in natural language processing (i.e., the standard, in-context, and chain-of-thought prompting). We then present a difficult sample selection procedure, taking into account sample diversity and uncertainty, to further challenge MLLMs equipped with the respective optimal prompting systems. We assess three open-source and one closed-source MLLMs on several visual attributes of image quality (e.g., structural and textural distortions, geometric transformations, and color differences) in both full-reference and no-reference scenarios. Experimental results show that only the closed-source GPT-4V provides a reasonable account for human perception of image quality, but is weak at discriminating fine-grained quality variations (e.g., color differences) and at comparing visual quality of multiple images, tasks humans can perform effortlessly.