mlsanityData to 7 Oct 2026

Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models

arXiv 2312.08962 ↗First posted 14 Dec 2023cs.CVcited by 104

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

We introduce a Depicted image Quality Assessment method (DepictQA), overcoming the constraints of traditional score-based methods. DepictQA allows for detailed, language-based, human-like evaluation of image quality by leveraging Multi-modal Large Language Models (MLLMs). Unlike conventional Image Quality Assessment (IQA) methods relying on scores, DepictQA interprets image content and distortions descriptively and comparatively, aligning closely with humans' reasoning process. To build the DepictQA model, we establish a hierarchical task framework, and collect a multi-modal IQA training dataset. To tackle the challenges of limited training data and multi-image processing, we propose to use multi-source training data and specialized image tags. These designs result in a better performance of DepictQA than score-based approaches on multiple benchmarks. Moreover, compared with general MLLMs, DepictQA can generate more accurate reasoning descriptive languages. We also demonstrate that our full-reference dataset can be extended to non-reference applications. These results showcase the research potential of multi-modal IQA methods. Codes and datasets are available in https://depictqa.github.io.

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 61 of its 91 references that are in the map. First posted on arXiv 14 Dec 2023.

Builds on

61 of 91 references.

References · 61

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2022Training language models to follow instructions with human feedbackcited by 15,712
  7. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  9. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  10. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  11. 2023Visual Instruction Tuningcited by 7,792
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
38 more
  1. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  2. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  3. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  4. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  5. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  7. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,037
  8. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  9. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  10. 2019Towards VQA Models That Can Readcited by 1,696
  11. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  12. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  13. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  14. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  15. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  16. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  17. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
  18. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  19. 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
  20. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 548
  21. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  22. 2018nocaps: novel object captioning at scalecited by 444
  23. 2023InternLM-XComposer: A Vision-Language Large Model for Advanced Text-image Comprehension and Compositioncited by 297
  24. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
  25. 2023MultiModal-GPT: A Vision and Language Model for Dialogue with Humanscited by 255
  26. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  27. 2016Deep Neural Networks for No-Reference and Full-Reference Image Quality Assessmentcited by 224
  28. 2021VisualGPT: Data-efficient Adaptation of Pretrained Language Models for Image Captioningcited by 190
  29. 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 177
  30. 2023MMICL: Empowering Vision-language Model with Multi-Modal In-Context Learningcited by 171
  31. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  32. 2018PieAPP: Perceptual Image-Error Assessment through Pairwise Preferencecited by 158
  33. 2023Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Modelscited by 152
  34. 2017RankIQA: Learning from Rankings for No-reference Image Quality Assessmentcited by 150
  35. 2020MetaIQA: Deep Meta-learning for No-Reference Image Quality Assessmentcited by 144
  36. 2016Learning a No-Reference Quality Metric for Single-Image Super-Resolutioncited by 143
  37. 2014Material Recognition in the Wild with the Materials in Context Databasecited by 77
  38. 2023A Comparative Study of Image Restoration Networks for General Backbone Network Designcited by 56

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 22; 2025: 42; 2026: 40.

104 papers.

Sort:

By area:

  1. 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 139
  2. 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
  3. 2025VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rankcited by 77
  4. 2024Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Comparecited by 66
  5. 2024DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curationcited by 64
  6. 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessmentcited by 43
  7. 2024XPSR: Cross-modal Priors for Diffusion-based Image Super-Resolutioncited by 41
  8. 2024An Intelligent Agentic System for Complex Image Restoration Problemscited by 41
  9. 2024Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Datasetcited by 38
  10. 2025ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understandingcited by 36
  11. 2024Quality-Aware Image-Text Alignment for Opinion-Unaware Image Quality Assessmentcited by 31
  12. 2024FineVQ: Fine-Grained User Generated Content Video Quality Assessmentcited by 28
38 more
  1. 2024Quality Assessment in the Era of Large Models: A Surveycited by 26
  2. 2025VQ-Insight: Teaching VLMs for AI-Generated Video Quality Understanding via Progressive Visual Reinforcement Learningcited by 26
  3. 20242AFC Prompting of Large Multimodal Models for Image Quality Assessmentcited by 20
  4. 2024LMM-PCQA: Assisting Point Cloud Quality Assessment with LMMcited by 19
  5. 2025UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texturecited by 19
  6. 2024SynArtifact: Classifying and Alleviating Artifacts in Synthetic Images via Vision-Language Modelcited by 15
  7. 2024VQA2: Visual Question Answering for Video Quality Assessmentcited by 15
  8. 2024PromptIQA: Boosting the Performance and Generalization for No-Reference Image Quality Assessment via Promptscited by 14
  9. 2024SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuningcited by 13
  10. 2025Q-SiT: Teaching LMMs for Image Quality Scoring and Interpretingcited by 13
  11. 2024Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessmentcited by 12
  12. 2024HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluatorcited by 12
  13. 2025Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Modelcited by 11
  14. 2025NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: KwaiSR Dataset and Studycited by 11
  15. 2025Self-Evolving Vision-Language Models for Image Quality Assessment via Voting and Rankingcited by 10
  16. 2025MedQ-Bench: Evaluating and Exploring Medical Image Quality Assessment Abilities in MLLMscited by 9
  17. 2025FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Modelcited by 9
  18. 2025RadarQA: Multi-modal Quality Analysis of Weather Radar Forecastscited by 8
  19. 2025PhotoFramer: Multi-modal Image Composition Instructioncited by 8
  20. 2026Generative World Renderercited by 8
  21. 2025Revisiting MLLM Based Image Quality Assessment: Errors and Remedycited by 7
  22. 2025MRI Super-Resolution with Deep Learning: A Comprehensive Surveycited by 7
  23. 2024DCA-Bench: A Benchmark for Dataset Curation Agentscited by 6
  24. 2025Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuningcited by 6
  25. 2025Gamma: Toward Generic Image Assessment with Mixture of Assessment Expertscited by 5
  26. 2025Scaling-up Perceptual Video Quality Assessmentcited by 5
  27. 2025RealSR-R1: Reinforcement Learning for Real-World Image Super-Resolution with Vision-Language Chain-of-Thoughtcited by 5
  28. 2025UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancementcited by 5
  29. 2024An Ensemble Approach to Short-form Video Quality Assessment Using Multimodal LLMcited by 4
  30. 2025Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Derainingcited by 4
  31. 2025Restore-R1: Efficient Image Restoration Agents via Reinforcement Learning with Multimodal LLM Perceptual Feedbackcited by 4
  32. 2024Quality Prediction of AI Generated Images and Videos: Emerging Trends and Opportunitiescited by 3
  33. 2025Position: Agentic Systems Constitute a Key Component of Next-Generation Intelligent Image Processingcited by 3
  34. 2025HandEval: Taking the First Step Towards Hand Quality Evaluation in Generated Imagescited by 3
  35. 2025Investigate the Low-level Visual Perception in Vision-Language based Image Quality Assessmentcited by 3
  36. 2026TIR-Agent: Training an Explorative and Efficient Agent for Image Restorationcited by 3
  37. 2026Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Executioncited by 3
  38. 2024U-Sketch: An Efficient Approach for Sketch to Image Diffusion Modelscited by 2

50 most cited of 104 All 104 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment
  2. 2026SIQA: Toward Reliable Scientific Image Quality Assessment
  3. 2026Q-Tacit: Image Quality Assessment via Latent Visual Reasoning
  4. 2025iDETEX: Empowering MLLMs for Intelligent DETailed EXplainable IQA
  5. 2026NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)
  6. 2026IQA-T1: Tool-based Visual Evidence Reasoning for Image Quality Assessment
  7. 2026Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Framework
  8. 2024Towards Open-ended Visual Quality Comparison