mlsanityData to 7 Oct 2026

A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment

arXiv 2403.10854 ↗First posted 16 Mar 2024cs.CVcited by 43

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

While Multimodal Large Language Models (MLLMs) have experienced significant advancement in visual understanding and reasoning, their potential to serve as powerful, flexible, interpretable, and text-driven models for Image Quality Assessment (IQA) remains largely unexplored. In this paper, we conduct a comprehensive and systematic study of prompting MLLMs for IQA. We first investigate nine prompting systems for MLLMs as the combinations of three standardized testing procedures in psychophysics (i.e., the single-stimulus, double-stimulus, and multiple-stimulus methods) and three popular prompting strategies in natural language processing (i.e., the standard, in-context, and chain-of-thought prompting). We then present a difficult sample selection procedure, taking into account sample diversity and uncertainty, to further challenge MLLMs equipped with the respective optimal prompting systems. We assess three open-source and one closed-source MLLMs on several visual attributes of image quality (e.g., structural and textural distortions, geometric transformations, and color differences) in both full-reference and no-reference scenarios. Experimental results show that only the closed-source GPT-4V provides a reasonable account for human perception of image quality, but is weak at discriminating fine-grained quality variations (e.g., color differences) and at comparing visual quality of multiple images, tasks humans can perform effortlessly.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 55 of its 77 references that are in the map. First posted on arXiv 16 Mar 2024.

Builds on

55 of 77 references.

Closest ancestors · 2

Same direction, newest first

  1. 20242AFC Prompting of Large Multimodal Models for Image Quality Assessment2 Feb
  2. 2023Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models14 Dec

Other references · 53

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  8. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  9. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  10. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  11. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  12. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
38 more
  1. 2023Visual Instruction Tuningcited by 7,792
  2. 2020Scaling Laws for Neural Language Modelscited by 6,411
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  4. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  5. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,878
  6. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,747
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  8. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  9. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  10. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  11. 2023A Survey on Multimodal Large Language Modelscited by 898
  12. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  13. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  14. 2023The Dawn of LMMs: Preliminary Explorations with GPT-4V(ision)cited by 677
  15. 2023An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuningcited by 622
  16. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  17. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  18. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 514
  19. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
  20. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  21. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
  22. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  23. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  24. 2019From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Qualitycited by 221
  25. 2023TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessmentcited by 220
  26. 2023EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizerscited by 189
  27. 2023Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Modelscited by 152
  28. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  29. 2023Understanding Catastrophic Forgetting in Language Models via Implicit Inferencecited by 138
  30. 2020Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wildcited by 134
  31. 2023Iterative Prompt Learning for Unsupervised Backlit Image Enhancementcited by 107
  32. 2024Towards Open-ended Visual Quality Comparisoncited by 85
  33. 2021Consequences of Misaligned AIcited by 79
  34. 2022On the Effect of Pretraining Corpora on In-context Learning by a Large-scale Language Modelcited by 78
  35. 2022Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Networkcited by 57
  36. 2023Assessor360: Multi-sequence Network for Blind Omnidirectional Image Quality Assessmentcited by 17
  37. 2022Measuring Perceptual Color Differences of Smartphone Photographscited by 8
  38. 2023Learning a Deep Color Difference Metric for Photographic Imagescited by 6

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 23; 2026: 6.

43 papers.

Sort:

By area:

  1. 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 139
  2. 2025VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rankcited by 77
  3. 2024Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Comparecited by 66
  4. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 47
  5. 2024Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMscited by 39
  6. 2024Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Datasetcited by 38
  7. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  8. 2024AutoVFX: Physically Realistic Video Editing from Natural Language Instructionscited by 27
  9. 2025Visual Text Processing: A Comprehensive Review and Unified Evaluationcited by 18
  10. 2024The Synergy between Data and Multi-Modal Large Language Models: A Survey from Co-Development Perspectivecited by 14
  11. 2024Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessmentcited by 12
  12. 2024HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluatorcited by 12
31 more
  1. 2025Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Modelcited by 11
  2. 2025Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Rankingcited by 10
  3. 2024From Text to Pixel: Advancing Long-Context Understanding in MLLMscited by 9
  4. 2024Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessmentcited by 9
  5. 2025MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMscited by 9
  6. 2025PocketSR: The Super-Resolution Expert in Your Pocket Mobilescited by 8
  7. 2025Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuningcited by 6
  8. 2024Modification Takes Courage: Seamless Image Stitching via Reference-Driven Inpaintingcited by 4
  9. 2024An Ensemble Approach to Short-form Video Quality Assessment Using Multimodal LLMcited by 4
  10. 2025REVEAL: Reasoning-Enhanced Forensic Evidence Analysis for Explainable AI-Generated Image Detectioncited by 4
  11. 2024Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunitiescited by 3
  12. 2024Enhancing Explainability in Multimodal Large Language Models Using Ontological Contextcited by 2
  13. 2025DGIQA: Depth-guided Feature Attention and Refinement for Generalizable Image Quality Assessmentcited by 2
  14. 2025DeQA-Doc: Adapting DeQA-Score to Document Image Quality Assessmentcited by 2
  15. 2025Creative4U: MLLMs-based Advertising Creative Image Selector with Comparative Reasoningcited by 2
  16. 2025Enhancing Diffusion-based Restoration Models via Difficulty-Adaptive Reinforcement Learning with IQA Rewardcited by 2
  17. 2026QualiRAG: Retrieval-Augmented Generation for Visual Quality Understandingcited by 2
  18. 2025Context and Pixel Aware Large Language Model for Video Quality Assessmentcited by 1
  19. 2025Taylor expansion-based Kolmogorov-Arnold network for blind image quality assessmentcited by 1
  20. 2025HiRQA: Hierarchical Ranking and Quality Alignment for Opinion-Unaware Image Quality Assessmentcited by 1
  21. 2026Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Frameworkcited by 1
  22. 2026Q-Hawkeye: Reliable Visual Policy Optimization for Image Quality Assessmentcited by 1
  23. 2026IQA-Spider: Unifying Multi-Granularity Image Quality Assessment with Reasoning, Grounding and Referringcited by 1
  24. 2026Tool-IQA: Augmenting Image Quality Assessment with Simple Toolscited by 1
  25. 2025The Devil is in the Darkness: Diffusion-Based Nighttime Dehazing Anchored in Brightness Perceptioncited by 0
  26. 2025Score2Instruct: Scaling Up Video Quality-Centric Instructions via Automated Dimension Scoringcited by 0
  27. 2025Parameter-Efficient Adaptation of mPLUG-Owl2 via Pixel-Level Visual Prompts for NR-IQAcited by 0
  28. 2025VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Resultscited by 0
  29. 2025Predictive Quality Assessment for Mobile Secure Graphicscited by 0
  30. 2025The Unanticipated Asymmetry Between Perceptual Optimization and Assessmentcited by 0
  31. 2026Scientific Image Quality Assessment via Multi-modal Retrieval-Augmented Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Revisiting MLLM Based Image Quality Assessment: Errors and Remedy
  2. 2025Evaluating Graphical Perception with Multimodal LLMs
  3. 2023Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models
  4. 2025Demystifying the Visual Quality Paradox in Multimodal Large Language Models
  5. 2025iDETEX: Empowering MLLMs for Intelligent DETailed EXplainable IQA
  6. 2026NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)
  7. 2024UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment
  8. 2024Visual Prompting in Multimodal Large Language Models: A Survey