mlsanityData to 7 Oct 2026

Towards Open-ended Visual Quality Comparison

arXiv 2402.16641 ↗First posted 26 Feb 2024cs.CVcited by 85

Not on the map. Why?

Cited by 37 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Comparative settings (e.g. pairwise choice, listwise ranking) have been adopted by a wide range of subjective studies for image quality assessment (IQA), as it inherently standardizes the evaluation criteria across different observers and offer more clear-cut responses. In this work, we extend the edge of emerging large multi-modality models (LMMs) to further advance visual quality comparison into open-ended settings, that 1) can respond to open-range questions on quality comparison; 2) can provide detailed reasonings beyond direct answers. To this end, we propose the Co-Instruct. To train this first-of-its-kind open-source open-ended visual quality comparer, we collect the Co-Instruct-562K dataset, from two sources: (a) LLM-merged single image quality description, (b) GPT-4V "teacher" responses on unlabeled data. Furthermore, to better evaluate this setting, we propose the MICBench, the first benchmark on multi-image comparison for LMMs. We demonstrate that Co-Instruct not only achieves in average 30% higher accuracy than state-of-the-art open-source LMMs, but also outperforms GPT-4V (its teacher), on both existing related benchmarks and the proposed MICBench. Our model is published at https://huggingface.co/q-future/co-instruct.

Key facts. Cited by 85 papers in the map (data through 7 Oct 2026). Builds on 47 of its 68 references that are in the map. First posted on arXiv 26 Feb 2024.

Builds on

47 of 68 references.

References · 47

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  7. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  8. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  9. 2015VQA: Visual Question Answeringcited by 2,489
  10. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  11. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  12. 2024Mixtral of Expertscited by 1,785
35 more
  1. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,447
  2. 2023mPLUG-Owl: Modularization Empowers Large Language Models with Multimodalitycited by 1,038
  3. 2023ShareGPT4V: Improving Large Multi-Modal Models with Better Captionscited by 1,000
  4. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 702
  5. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  6. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
  7. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  8. 2019KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessmentcited by 351
  9. 2019Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Comingcited by 332
  10. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  11. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  12. 2023Improving Text Embeddings with Large Language Modelscited by 295
  13. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
  14. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  15. 2019From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Qualitycited by 221
  16. 2015Massive Online Crowdsourced Study of Subjective and Objective Picture Qualitycited by 208
  17. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  18. 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 177
  19. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  20. 2021No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistencycited by 165
  21. 2019Quality Assessment of In-the-Wild Videoscited by 164
  22. 2018PieAPP: Perceptual Image-Error Assessment through Pairwise Preferencecited by 158
  23. 2023Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Modelscited by 152
  24. 2017RankIQA: Learning from Rankings for No-reference Image Quality Assessmentcited by 150
  25. 2019RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolutioncited by 145
  26. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  27. 2020Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wildcited by 134
  28. 2019dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairscited by 73
  29. 2019Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rankcited by 52
  30. 2023Subjective and Objective Quality Assessment for in-the-Wild Computer Graphics Imagescited by 21
  31. 20242AFC Prompting of Large Multimodal Models for Image Quality Assessmentcited by 20
  32. 2020Active Sampling for Pairwise Comparisons via Approximate Message Passing and Information Gain Maximizationcited by 17
  33. 2020Subjective Quality Assessment for YouTube UGC Datasetcited by 11
  34. 2018Hybrid-MST: A Hybrid Active Sampling Strategy for Pairwise Preference Aggregationcited by 10
  35. 2022Exploring and Evaluating Image Restoration Potential in Dynamic Scenescited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 29; 2025: 39; 2026: 17.

85 papers.

Sort:

By area:

  1. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  2. 2024MANTIS: Interleaved Multi-Image Instruction Tuningcited by 212
  3. 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 139
  4. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  5. 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
  6. 2024MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMscited by 89
  7. 2024MM1.5: Methods, Analysis & Insights from Multimodal LLM Fine-tuningcited by 75
  8. 2024Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Comparecited by 66
  9. 2024AesExpert: Towards Multi-modality Foundation Model for Image Aesthetics Perceptioncited by 53
  10. 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessmentcited by 43
  11. 2024An Intelligent Agentic System for Complex Image Restoration Problemscited by 41
  12. 2024Subjective-Aligned Dataset and Metric for Text-to-Video Quality Assessmentcited by 39
38 more
  1. 2024Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Datasetcited by 38
  2. 2024MC-Bench: A Benchmark for Multi-Context Visual Grounding in the Era of MLLMscited by 38
  3. 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
  4. 2025VLM2-Bench: A Closer Look at How Well VLMs Implicitly Link Explicit Matching Visual Cuescited by 33
  5. 2024OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generationcited by 30
  6. 2024Quality Assessment in the Era of Large Models: A Surveycited by 26
  7. 2024Large Multi-modality Model Assisted AI-Generated Image Quality Assessmentcited by 24
  8. 2024NTIRE 2024 Quality Assessment of AI-Generated Content Challengecited by 20
  9. 2024LMM-PCQA: Assisting Point Cloud Quality Assessment with LMMcited by 19
  10. 2025UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texturecited by 19
  11. 2024VQA2: Visual Question Answering for Video Quality Assessmentcited by 15
  12. 2024Video Quality Assessment: A Comprehensive Surveycited by 15
  13. 2025Hybrid Agents for Image Restorationcited by 14
  14. 2025Q-SiT: Teaching LMMs for Image Quality Scoring and Interpretingcited by 13
  15. 2024Grounding-IQA: Grounding Multimodal Language Model for Image Quality Assessmentcited by 12
  16. 2024HEIE: MLLM-Based Hierarchical Explainable AIGC Image Implausibility Evaluatorcited by 12
  17. 2025The Photographer Eye: Teaching Multimodal Large Language Models to Understand Image Aesthetics like Photographerscited by 12
  18. 2025NTIRE 2025 Challenge on Short-form UGC Video Quality Assessment and Enhancement: KwaiSR Dataset and Studycited by 11
  19. 2024Dog-IQA: Standard-guided Zero-shot MLLM for Mix-grained Image Quality Assessmentcited by 9
  20. 2024THQA: A Perceptual Quality Assessment Database for Talking Headscited by 8
  21. 2024G-Refine: A General Quality Refiner for Text-to-Image Generationcited by 8
  22. 2025RadarQA: Multi-modal Quality Analysis of Weather Radar Forecastscited by 8
  23. 2024How is Visual Attention Influenced by Text Guidance? Database and Modelcited by 7
  24. 2025Fine-grained Image Quality Assessment for Perceptual Image Restorationcited by 7
  25. 2024Light-VQA+: A Video Quality Assessment Model for Exposure Correction with Vision-Language Guidancecited by 6
  26. 2025T2VEval: Benchmark Dataset and Objective Evaluation Method for T2V-generated Videoscited by 6
  27. 2025UVE: Are MLLMs Unified Evaluators for AI-Generated Videos?cited by 6
  28. 2025Q-Adapt: Adapting LMM for Visual Quality Assessment with Progressive Instruction Tuningcited by 6
  29. 2025Scaling-up Perceptual Video Quality Assessmentcited by 5
  30. 2025UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancementcited by 5
  31. 2024An Ensemble Approach to Short-form Video Quality Assessment Using Multimodal LLMcited by 4
  32. 2025Restore-R1: Efficient Image Restoration Agents via Reinforcement Learning with Multimodal LLM Perceptual Feedbackcited by 4
  33. 2025Position: Agentic Systems Constitute a Key Component of Next-Generation Intelligent Image Processingcited by 3
  34. 2024Towards Unified Benchmark and Models for Multi-Modal Perceptual Metricscited by 2
  35. 2025DeQA-Doc: Adapting DeQA-Score to Document Image Quality Assessmentcited by 2
  36. 2025MIHBench: Benchmarking and Mitigating Multi-Image Hallucinations in Multimodal Large Language Modelscited by 2
  37. 2025Refine-IQA: Multi-Stage Reinforcement Finetuning for Perceptual Image Quality Assessmentcited by 2
  38. 2025ViDA-UGC: Detailed Image Quality Analysis via Visual Distortion Assessment for UGC Imagescited by 2

50 most cited of 85 All 85 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024VisualCritic: Making LMMs Perceive Visual Quality Like Humans
  2. 2024Q-Ground: Image Quality Grounding with Large Multi-modality Models
  3. 2026Enhancing Image Quality Assessment Ability of LMMs via Retrieval-Augmented Generation
  4. 2024Mitigating Perception Bias: A Training-Free Approach to Enhance LMM for Image Quality Assessment
  5. 2023Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models
  6. 2026NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)
  7. 2024UniQA: Unified Vision-Language Pre-training for Image Quality and Aesthetic Assessment
  8. 2026Evaluating Vision-Language Models for Image Quality Assessment using Psychophysical Data