# VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models: Methods and Results

arXiv: https://arxiv.org/abs/2509.09190 · HTML: https://mlsanity.com/paper/2509.09190/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 37 of its 64 references that are in the map.
- First posted on arXiv 11 Sep 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Image editing and quality assessment](https://mlsanity.com/mllm/image-editing-and-quality-assessment/)

## Builds on (most cited first, 30 of 37)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels](https://mlsanity.com/paper/2312.17090/) (2023)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [From Patches to Pictures (PaQ-2-PiQ): Mapping the Perceptual Space of Picture Quality](https://mlsanity.com/paper/1912.10088/) (2019)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [RankIQA: Learning from Rankings for No-reference Image Quality Assessment](https://mlsanity.com/paper/1707.08347/) (2017)
- [Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Models](https://mlsanity.com/paper/2311.06783/) (2023)
- [Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution](https://mlsanity.com/paper/2501.11561/) (2025)
- [Uncertainty-Aware Blind Image Quality Assessment in the Laboratory and Wild](https://mlsanity.com/paper/2005.13983/) (2020)
- [Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models](https://mlsanity.com/paper/2312.08962/) (2023)
- [Q-Insight: Understanding Image Quality via Visual Reinforcement Learning](https://mlsanity.com/paper/2503.22679/) (2025)
- [Towards Open-ended Visual Quality Comparison](https://mlsanity.com/paper/2402.16641/) (2024)
- [VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rank](https://mlsanity.com/paper/2505.14460/) (2025)
- [Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Compare](https://mlsanity.com/paper/2405.19298/) (2024)
- [A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessment](https://mlsanity.com/paper/2403.10854/) (2024)
- [Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltales](https://mlsanity.com/paper/2107.05821/) (2021)
- [Pixel-Inconsistency Modeling for Image Manipulation Localization](https://mlsanity.com/paper/2310.00234/) (2023)
- [MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection](https://mlsanity.com/paper/2404.08452/) (2024)
- [DeepWSD: Projecting Degradations in Perceptual Space to Wasserstein Distance in Deep Feature Space](https://mlsanity.com/paper/2208.03323/) (2022)
- [2AFC Prompting of Large Multimodal Models for Image Quality Assessment](https://mlsanity.com/paper/2402.01162/) (2024)
- [DeepDC: Deep Distance Correlation as a Perceptual Image Quality Evaluator](https://mlsanity.com/paper/2211.04927/) (2022)
- [MSTRIQ: No Reference Image Quality Assessment Based on Swin Transformer with Multi-Stage Fusion](https://mlsanity.com/paper/2205.10101/) (2022)
- [Gap-closing Matters: Perceptual Quality Evaluation and Optimization of Low-Light Image Enhancement](https://mlsanity.com/paper/2302.11464/) (2023)
- [Unrolled Decomposed Unpaired Learning for Controllable Low-Light Video Enhancement](https://mlsanity.com/paper/2408.12316/) (2024)

## Abstract (arXiv)

This paper presents a summary of the VQualA 2025 Challenge on Visual Quality Comparison for Large Multimodal Models (LMMs), hosted as part of the ICCV 2025 Workshop on Visual Quality Assessment. The challenge aims to evaluate and enhance the ability of state-of-the-art LMMs to perform open-ended and detailed reasoning about visual quality differences across multiple images. To this end, the competition introduces a novel benchmark comprising thousands of coarse-to-fine grained visual quality comparison tasks, spanning single images, pairs, and multi-image groups. Each task requires models to provide accurate quality judgments. The competition emphasizes holistic evaluation protocols, including 2AFC-based binary preference and multi-choice questions (MCQs). Around 100 participants submitted entries, with five models demonstrating the emerging capabilities of instruction-tuned LMMs on quality assessment. This challenge marks a significant step toward open-domain visual quality reasoning and comparison and serves as a catalyst for future research on interpretable and human-aligned quality evaluation systems.