mlsanityData to 14 Oct 2026

NTIRE 2025 challenge on Text to Image Generation Model Quality Assessment

arXiv 2505.16314 ↗First posted 22 May 2025cs.CV cs.AIcited by 8

Not on the map. Why?

Abstract (arXiv)

This paper reports on the NTIRE 2025 challenge on Text to Image (T2I) generation model quality assessment, which will be held in conjunction with the New Trends in Image Restoration and Enhancement Workshop (NTIRE) at CVPR 2025. The aim of this challenge is to address the fine-grained quality assessment of text-to-image generation models. This challenge evaluates text-to-image models from two aspects: image-text alignment and image structural distortion detection, and is divided into the alignment track and the structural track. The alignment track uses the EvalMuse-40K, which contains around 40K AI-Generated Images (AIGIs) generated by 20 popular generative models. The alignment track has a total of 371 registered participants. A total of 1,883 submissions are received in the development phase, and 507 submissions are received in the test phase. Finally, 12 participating teams submitted their models and fact sheets. The structure track uses the EvalMuse-Structure, which contains 10,000 AI-Generated Images (AIGIs) with corresponding structural distortion mask. A total of 211 participants have registered in the structure track. A total of 1155 submissions are received in the development phase, and 487 submissions are received in the test phase. Finally, 8 participating teams submitted their models and fact sheets. Almost all methods have achieved better results than baseline methods, and the winning methods in both tracks have demonstrated superior prediction performance on T2I model quality assessment.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 61 of its 80 references that are in the map. First posted on arXiv 22 May 2025.

Builds on

61 of 80 references.

References · 61

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  10. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  11. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  12. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
38 more
  1. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  2. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  3. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 691
  4. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  5. 2023mPLUG-Owl2: Revolutionizing Multi-modal Large Language Model with Modality Collaborationcited by 547
  6. 2022MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessmentcited by 522
  7. 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
  8. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  9. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  10. 2023Florence-2: Advancing a Unified Representation for a Variety of Vision Taskscited by 410
  11. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 379
  12. 2019Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Networkcited by 302
  13. 2023Human Preference Score: Better Aligning Text-to-Image Models with Human Preferencecited by 287
  14. 2023Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspectivecited by 280
  15. 2022DETRs with Collaborative Hybrid Assignments Trainingcited by 274
  16. 2025Unified Reward Model for Multimodal Understanding and Generationcited by 199
  17. 2024A Sanity Check for AI-generated Image Detectioncited by 189
  18. 2024Ovis: Structural Embedding Alignment for Multimodal Large Language Modelcited by 169
  19. 2023AGIQA-3K: An Open Database for AI-Generated Image Quality Assessmentcited by 142
  20. 2023Rich Human Feedback for Text-to-Image Generationcited by 141
  21. 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 141
  22. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
  23. 2022AltCLIP: Altering the Language Encoder in CLIP for Extended Language Capabilitiescited by 85
  24. 2021Blind Quality Assessment for in-the-Wild Images via Hierarchical Feature Fusion and Iterative Mixed Database Trainingcited by 71
  25. 2024SAM2-UNet: Segment Anything 2 Makes Strong Encoder for Natural and Medical Image Segmentationcited by 64
  26. 2023AIGCIQA2023: A Large-scale Image Quality Assessment Database for AI Generated Images: from the Perspectives of Quality, Authenticity and Correspondencecited by 61
  27. 2024The Ninth NTIRE 2024 Efficient Super-Resolution Challenge Reportcited by 57
  28. 2023A Perceptual Quality Assessment Exploration for AIGC Imagescited by 55
  29. 2024Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratingscited by 44
  30. 2024NTIRE 2024 Challenge on Image Super-Resolution (x4): Methods and Resultscited by 40
  31. 2024EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluationcited by 39
  32. 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
  33. 2024NTIRE 2024 Challenge on Short-form UGC Video Quality Assessment: Methods and Resultscited by 29
  34. 2025The Tenth NTIRE 2025 Efficient Super-Resolution Challenge Reportcited by 26
  35. 2025NTIRE 2025 Challenge on Cross-Domain Few-Shot Object Detection: Methods and Resultscited by 25
  36. 2025NTIRE 2025 Challenge on Day and Night Raindrop Removal for Dual-Focused Images: Methods and Resultscited by 22
  37. 2025The Tenth NTIRE 2025 Image Denoising Challenge Reportcited by 21
  38. 2021Study Group Learning: Improving Retinal Vessel Segmentation Trained with Noisy Labelscited by 20

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 8.

8 papers.

  1. 2025NTIRE 2025 Challenge on Video Quality Enhancement for Video Conferencing: Datasets, Methods and Resultscited by 9
  2. 2025NTIRE 2025 Challenge on RAW Image Restoration and Super-Resolutioncited by 9
  3. 2025NTIRE 2025 XGC Quality Assessment Challenge: Methods and Resultscited by 6
  4. 2025NTIRE 2025 the 2nd Restore Any Image Model (RAIM) in the Wild Challengecited by 5
  5. 2025RAW Image Reconstruction from RGB on Smartphones. NTIRE 2025 Challenge Reportcited by 4
  6. 2025NTIRE 2025 Challenge on HR Depth from Images of Specular and Transparent Surfacescited by 3
  7. 2025NTIRE 2025 Image Shadow Removal Challenge Reportcited by 1
  8. 2025NTIRE 2025 Challenge on Low Light Image Enhancement: Methods and Resultscited by 1