# Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Dataset

arXiv: https://arxiv.org/abs/2405.18842 · HTML: https://mlsanity.com/paper/2405.18842/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 72 of its 125 references that are in the map.
- First posted on arXiv 29 May 2024.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 30 of 72)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering](https://mlsanity.com/paper/2209.09513/) (2022)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AI](https://mlsanity.com/paper/1910.10045/) (2019)
- [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://mlsanity.com/paper/2203.10244/) (2022)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [DocVQA: A Dataset for VQA on Document Images](https://mlsanity.com/paper/2007.00398/) (2020)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Transformer Interpretability Beyond Attention Visualization](https://mlsanity.com/paper/2012.09838/) (2020)

## Built on by (most cited first, 30 of 38)

- [Q-Insight: Understanding Image Quality via Visual Reinforcement Learning](https://mlsanity.com/paper/2503.22679/) (2025)
- [An Intelligent Agentic System for Complex Image Restoration Problems](https://mlsanity.com/paper/2410.17809/) (2024)
- [JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration](https://mlsanity.com/paper/2504.04158/) (2025)
- [FineVQ: Fine-Grained User Generated Content Video Quality Assessment](https://mlsanity.com/paper/2412.19238/) (2024)
- [Quality Assessment in the Era of Large Models: A Survey](https://mlsanity.com/paper/2409.00031/) (2024)
- [Reasoning as Representation: Rethinking Visual Reinforcement Learning in Image Quality Assessment](https://mlsanity.com/paper/2510.11369/) (2025)
- [RBench-V: A Primary Assessment for Visual Reasoning Models with Multi-modal Outputs](https://mlsanity.com/paper/2505.16770/) (2025)
- [Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment](https://mlsanity.com/paper/2506.05384/) (2025)
- [SEAGULL: No-reference Image Quality Assessment for Regions of Interest via Vision-Language Instruction Tuning](https://mlsanity.com/paper/2411.10161/) (2024)
- [Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessment](https://mlsanity.com/paper/2411.17237/) (2024)
- [Next Token Is Enough: Realistic Image Quality and Aesthetic Scoring with Multimodal Large Language Model](https://mlsanity.com/paper/2503.06141/) (2025)
- [Generative World Renderer](https://mlsanity.com/paper/2604.02329/) (2026)
- [Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessment](https://mlsanity.com/paper/2410.02505/) (2024)
- [PhotoArtAgent: Intelligent Photo Retouching with Language Model-Based Artist Agents](https://mlsanity.com/paper/2505.23130/) (2025)
- [Revisiting MLLM Based Image Quality Assessment: Errors and Remedy](https://mlsanity.com/paper/2511.07812/) (2025)
- [Zoom-IQA: Image Quality Assessment with Reliable Region-Aware Reasoning](https://mlsanity.com/paper/2601.02918/) (2026)
- [Scaling-up Perceptual Video Quality Assessment](https://mlsanity.com/paper/2505.22543/) (2025)
- [TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration](https://mlsanity.com/paper/2603.27742/) (2026)
- [Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining](https://mlsanity.com/paper/2502.12600/) (2025)
- [Position: Agentic Systems Constitute a Key Component of Next-Generation Intelligent Image Processing](https://mlsanity.com/paper/2505.16007/) (2025)
- [Refine-IQA: Multi-Stage Reinforcement Finetuning for Perceptual Image Quality Assessment](https://mlsanity.com/paper/2508.03763/) (2025)
- [ViDA-UGC: Detailed Image Quality Analysis via Visual Distortion Assessment for UGC Images](https://mlsanity.com/paper/2508.12605/) (2025)
- [MVQA-68K: A Multi-dimensional and Causally-annotated Dataset with Quality Interpretability for Video Assessment](https://mlsanity.com/paper/2509.11589/) (2025)
- [AgenticIQA: An Agentic Framework for Adaptive and Interpretable Image Quality Assessment](https://mlsanity.com/paper/2509.26006/) (2025)
- [Enhancing Diffusion-based Restoration Models via Difficulty-Adaptive Reinforcement Learning with IQA Reward](https://mlsanity.com/paper/2511.01645/) (2025)
- [OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessment](https://mlsanity.com/paper/2510.10609/) (2025)
- [VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment](https://mlsanity.com/paper/2511.17962/) (2025)
- [Position: Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered](https://mlsanity.com/paper/2603.00643/) (2026)
- [IQA-Spider: Unifying Multi-Granularity Image Quality Assessment with Reasoning, Grounding and Referring](https://mlsanity.com/paper/2605.24553/) (2026)
- [LMM-IQA: Image Quality Assessment for Low-Dose CT Imaging](https://mlsanity.com/paper/2511.07298/) (2025)

## Abstract (arXiv)

With the rapid advancement of Vision Language Models (VLMs), VLM-based Image Quality Assessment (IQA) seeks to describe image quality linguistically to align with human expression and capture the multifaceted nature of IQA tasks. However, current methods are still far from practical usage. First, prior works focus narrowly on specific sub-tasks or settings, which do not align with diverse real-world applications. Second, their performance is sub-optimal due to limitations in dataset coverage, scale, and quality. To overcome these challenges, we introduce the enhanced Depicted image Quality Assessment model (DepictQA-Wild). Our method includes a multi-functional IQA task paradigm that encompasses both assessment and comparison tasks, brief and detailed responses, full-reference and non-reference scenarios. We introduce a ground-truth-informed dataset construction approach to enhance data quality, and scale up the dataset to 495K under the brief-detail joint framework. Consequently, we construct a comprehensive, large-scale, and high-quality dataset, named DQ-495K. We also retain image resolution during training to better handle resolution-related quality issues, and estimate a confidence score that is helpful to filter out low-quality responses. Experimental results demonstrate that DepictQA-Wild significantly outperforms traditional score-based methods, prior VLM-based IQA models, and proprietary GPT-4V in distortion identification, instant rating, and reasoning tasks. Our advantages are further confirmed by real-world applications including assessing the web-downloaded images and ranking model-processed images. Codes, datasets, and model weights have been released in https://depictqa.github.io/.