mlsanityData to 7 Oct 2026

Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levels

arXiv 2312.17090 ↗First posted 28 Dec 2023cs.CV cs.CL cs.LGcited by 489

Not on the map. Why?

Cited by 74 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The explosion of visual content available online underscores the requirement for an accurate machine assessor to robustly evaluate scores across diverse types of visual contents. While recent studies have demonstrated the exceptional potentials of large multi-modality models (LMMs) on a wide range of related fields, in this work, we explore how to teach them for visual rating aligned with human opinions. Observing that human raters only learn and judge discrete text-defined levels in subjective studies, we propose to emulate this subjective process and teach LMMs with text-defined rating levels instead of scores. The proposed Q-Align achieves state-of-the-art performance on image quality assessment (IQA), image aesthetic assessment (IAA), as well as video quality assessment (VQA) tasks under the original LMM structure. With the syllabus, we further unify the three tasks into one model, termed the OneAlign. In our experiments, we demonstrate the advantage of the discrete-level-based syllabus over direct-score-based variants for LMMs. Our code and the pre-trained weights are released at https://github.com/Q-Future/Q-Align.

Key facts. Cited by 489 papers in the map (data through 7 Oct 2026). Builds on 29 of its 44 references that are in the map. First posted on arXiv 28 Dec 2023.

Builds on

29 of 44 references.

References · 29

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  3. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  4. 2022CoCa: Contrastive Captioners are Image-Text Foundation Modelscited by 1,265
  5. 2021MUSIQ: Multi-scale Image Quality Transformercited by 1,058
  6. 2023Shikra: Unleashing Multimodal LLM's Referential Dialogue Magiccited by 891
  7. 2023LLaMA-Adapter V2: Parameter-Efficient Visual Instruction Modelcited by 602
  8. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 545
  9. 2017NIMA: Neural Image Assessmentcited by 387
  10. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  11. 2019Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Networkcited by 301
  12. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 276
17 more
  1. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 247
  2. 2022FAST-VQA: Efficient End-to-end Video Quality Assessment with Fragment Samplingcited by 179
  3. 2019Quality Assessment of In-the-Wild Videoscited by 164
  4. 2023Q-Instruct: Improving Low-level Visual Abilities for Multi-modality Foundation Modelscited by 152
  5. 2020Patch-VQ: 'Patching Up' the Video Quality Problemcited by 147
  6. 2020UGC-VQA: Benchmarking Blind Video Quality Assessment for User Generated Contentcited by 140
  7. 2022A Deep Learning based No-reference Quality Assessment Model for UGC Videoscited by 128
  8. 2016Photo Aesthetics Ranking Network with Attributes and Content Adaptationcited by 122
  9. 2021RAPIQUE: Rapid and Accurate Video Quality Prediction of User Generated Contentcited by 100
  10. 2021Blindly Assess Quality of In-the-Wild Videos via Quality-aware Pre-training and Motion Perceptioncited by 97
  11. 2023VILA: Learning Image Aesthetics from User Comments with Vision-Language Pretrainingcited by 63
  12. 2019Effective Aesthetics Prediction with Multi-level Spatially Pooled Featurescited by 52
  13. 2022Neighbourhood Representative Sampling for Efficient End-to-end Video Quality Assessmentcited by 50
  14. 2022DisCoVQA: Temporal Distortion-Content Transformers for Video Quality Assessmentcited by 41
  15. 2023Exploring Opinion-unaware Video Quality Assessment with Semantic Affinity Criterioncited by 23
  16. 2023Towards Robust Text-Prompted Semantic Criterion for In-the-Wild Video Quality Assessmentcited by 16
  17. 2019Aesthetic Image Captioning From Weakly-Labelled Photographscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 91; 2025: 206; 2026: 192.

489 papers.

Sort:

By area:

  1. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understandingcited by 644
  2. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  3. 2024WonderWorld: Interactive 3D Scene Generation from a Single Imagecited by 197
  4. 2025WorldScore: A Unified Evaluation Benchmark for World Generationcited by 143
  5. 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 139
  6. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  7. 2024MAmmoTH-VL: Eliciting Multimodal Reasoning with Instruction Tuning at Scalecited by 125
  8. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
  9. 2025Q-Insight: Understanding Image Quality via Visual Reinforcement Learningcited by 93
  10. 2025LongCat-Image Technical Reportcited by 92
  11. 2024LN3DIFF++: Scalable Latent Neural Fields Diffusion for Speedy 3D Generationcited by 91
  12. 2025HunyuanWorld 1.0: Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixelscited by 91
38 more
  1. 2024Towards Open-ended Visual Quality Comparisoncited by 85
  2. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 84
  3. 2024DreamScene360: Unconstrained Text-to-3D Scene Generation with Panoramic Gaussian Splattingcited by 81
  4. 2025VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rankcited by 77
  5. 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 77
  6. 2024Adaptive Image Quality Assessment via Teaching Large Multimodal Model to Comparecited by 66
  7. 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 64
  8. 2024AesBench: An Expert Benchmark for Multimodal Large Language Models on Image Aesthetics Perceptioncited by 63
  9. 20244K4DGen: Panoramic 4D Generation at 4K Resolutioncited by 54
  10. 2024Comp4D: LLM-Guided Compositional 4D Scene Generationcited by 50
  11. 2025VMBench: A Benchmark for Perception-Aligned Video Motion Generationcited by 48
  12. 2024A Comprehensive Study of Multimodal Large Language Models for Image Quality Assessmentcited by 43
  13. 2024Image Watermarks are Removable Using Controllable Regeneration from Clean Noisecited by 43
  14. 2025Distribution Matching Distillation Meets Reinforcement Learningcited by 43
  15. 2024An Intelligent Agentic System for Complex Image Restoration Problemscited by 41
  16. 2025StableAvatar: Infinite-Length Audio-Driven Avatar Video Generationcited by 41
  17. 2025One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillationcited by 40
  18. 2024A-Bench: Are LMMs Masters at Evaluating AI-generated Images?cited by 39
  19. 2024Q-Bench-Video: Benchmarking the Video Quality Understanding of LMMscited by 39
  20. 2024Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Datasetcited by 38
  21. 2024OnlineVPO: Align Video Diffusion Model with Online Video-Centric Preference Optimizationcited by 38
  22. 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 38
  23. 2025VideoScore2: Think before You Score in Generative Video Evaluationcited by 37
  24. 2025ArtiMuse: Fine-Grained Image Aesthetics Assessment with Joint Scoring and Expert-Level Understandingcited by 36
  25. 2025Stochastic Self-Guidance for Training-Free Enhancement of Diffusion Modelscited by 35
  26. 2026HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worldscited by 34
  27. 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
  28. 2024Scalable Ranked Preference Optimization for Text-to-Image Generationcited by 33
  29. 2025JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restorationcited by 32
  30. 2024Quality-Aware Image-Text Alignment for Opinion-Unaware Image Quality Assessmentcited by 31
  31. 2025WonderTurbo: Generating Interactive 3D World in 0.72 Secondscited by 31
  32. 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 30
  33. 2025FlashWorld: High-quality 3D Scene Generation within Secondscited by 30
  34. 2026Rethinking Video Generation Model for the Embodied Worldcited by 30
  35. 2024MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Modelcited by 29
  36. 2024NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Resultscited by 29
  37. 2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?cited by 29
  38. 2024LMM-VQA: Advancing Video Quality Assessment with Large Multimodal Modelscited by 28

50 most cited of 489 All 489 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024VisualCritic: Making LMMs Perceive Visual Quality Like Humans
  2. 2026Decoupling Perception and Calibration: Label-Efficient Image Quality Assessment Framework
  3. 2024Q-Ground: Image Quality Grounding with Large Multi-modality Models
  4. 2023Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models
  5. 2023Q-Boost: On Visual Quality Assessment Ability of Low-level Multi-Modality Foundation Models
  6. 2025Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment
  7. 2026VLM Judges Can Rank but Cannot Score: Task-Dependent Uncertainty in Multimodal Evaluation
  8. 2026Q-Tacit: Image Quality Assessment via Latent Visual Reasoning