VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Results
On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment
Abstract (arXiv)
This paper presents a summary of the VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models (LMMs), hosted as part of the ICCV 2025 Workshop on Visual Quality Assessment. The challenge aims to evaluate and enhance the ability of state-of-the-art LMMs to perform open-ended and detailed reasoning about visual quality differences across multiple images. To this end, the competition introduces a novel benchmark comprising thousands of coarse-to-fine grained visual quality comparison tasks, spanning single images, pairs, and multi-image groups. Each task requires models to provide accurate quality judgments. The competition emphasizes holistic evaluation protocols, including 2AFC-based binary preference and multi-choice questions (MCQs). Around 100 participants submitted entries, with five models demonstrating the emerging capabilities of instruction-tuned LMMs on quality assessment. This challenge marks a significant step toward open-domain visual quality reasoning and comparison and serves as a catalyst for future research on interpretable and human-aligned quality evaluation systems.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 64 references that are in the map. First posted on arXiv 11 Sep 2025.
Builds on
37 of 64 references.
Closest ancestors · 5
Same direction, newest first
- 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution20 Jan
- 2024Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare29 May
- 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment16 Mar
- 20242AFC Prompting of Large Multimodal Models for Image Quality Assessment2 Feb
- 2023Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models14 Dec
Other references · 32
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2024GPT-4o System Cardcited by 4,467
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
- 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 792
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2019Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Networkcited by 301
- 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
- 2019From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Qualitycited by 221
- 2018PieAPP: Perceptual Image-Error Assessment through Pairwise Preferencecited by 158
- 2023Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Modelscited by 152
20 more
- 2017RankIQA: Learning from Rankings for No-reference Image Quality Assessmentcited by 150
- 2020Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wildcited by 134
- 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
- 2024Towards Open-ended Visual Quality Comparisoncited by 85
- 2025VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rankcited by 77
- 2021Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltalescited by 39
- 2023Pixel-Inconsistency Modeling for Image Manipulation Localizationcited by 33
- 2024MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detectioncited by 27
- 2022DeepWSD: Projecting Degradations in Perceptual Space to Wasserstein Distance in Deep Feature Spacecited by 19
- 2022MSTRIQ: No Reference Image Quality Assessment Based on Swin Transformer with Multi-Stage Fusioncited by 10
- 2022DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluatorcited by 10
- 2023Gap-closing Matters: Perceptual Quality Evaluation and Optimization of Low-Light Image Enhancementcited by 7
- 2024Unrolled Decomposed Unpaired Learning for Controllable Low-Light Video Enhancementcited by 7
- 2024AI-generated Image Quality Assessment in Visual Communicationcited by 7
- 2025VQualA 2025 Challenge on Engagement Prediction for Short Videos: Methods and Resultscited by 6
- 2023Perceptual Quality Assessment of 360^circ Images Based on Generative Scanpath Representationcited by 5
- 2025VQualA 2025 Challenge on Image Super-Resolution Generated Content Quality Assessment: Methods and Resultscited by 5
- 2025VQualA 2025 Challenge on Face Image Quality Assessment: Methods and Resultscited by 4
- 2022Deep Feature Statistics Mapping for Generalized Screen Content Image Quality Assessmentcited by 3
- 2025Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligencecited by 3
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment
- 2024LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Models
- 2026LLaVA-Assessor: Building the Foundation LMM For Visual Quality Assessment
- 2026NTIRE 2026 The 3rd Restore Any Image Model (RAIM) Challenge: Professional Image Quality Assessment (Track 1)
- 2024VisualCritic: Making LMMs Perceive Visual Quality Like Humans
- 2024Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Dataset
- 2024VQA2: Visual Question Answering for Video Quality Assessment
- 2024Q-Ground: Image Quality Grounding with Large Multi-modality Models