# Q-Eval-100K: Evaluating Visual Quality and Alignment Level for Text-to-Vision Content

arXiv: https://arxiv.org/abs/2503.02357 · HTML: https://mlsanity.com/paper/2503.02357/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2025.

## Built on by (most cited first, 30 of 32)

- [A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deployment](https://mlsanity.com/paper/2504.15585/) (2025)
- [Q-Insight: Understanding Image Quality via Visual Reinforcement Learning](https://mlsanity.com/paper/2503.22679/) (2025)
- [Reasoning as Representation: Rethinking Visual Reinforcement Learning in Image Quality Assessment](https://mlsanity.com/paper/2510.11369/) (2025)
- [MMIG-Bench: Towards Comprehensive and Explainable Evaluation of Multi-Modal Image Generation Models](https://mlsanity.com/paper/2505.19415/) (2025)
- [LOVE: Benchmarking and Evaluating Text-to-Video Generation and Video-to-Text Interpretation](https://mlsanity.com/paper/2505.12098/) (2025)
- [RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts](https://mlsanity.com/paper/2508.12291/) (2025)
- [X2HDR: HDR Image Generation in a Perceptually Uniform Space](https://mlsanity.com/paper/2602.04814/) (2026)
- [NTIRE 2025 XGC Quality Assessment Challenge: Methods and Results](https://mlsanity.com/paper/2506.02875/) (2025)
- [UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancement](https://mlsanity.com/paper/2512.06750/) (2025)
- [LongT2IBench: A Benchmark for Evaluating Long Text-to-Image Generation with Graph-structured Annotations](https://mlsanity.com/paper/2512.09271/) (2025)
- [Fine-grained Image Aesthetic Assessment: Learning Discriminative Scores from Relative Ranks](https://mlsanity.com/paper/2603.03907/) (2026)
- [Engagement Prediction of Short Videos with Large Multimodal Models](https://mlsanity.com/paper/2508.02516/) (2025)
- [Towards Holistic Visual Quality Assessment of AI-Generated Videos: A LLM-Based Multi-Dimensional Evaluation Model](https://mlsanity.com/paper/2506.04715/) (2025)
- [What Makes a Good Generated Image? Investigating Human and Multimodal LLM Image Preference Alignment](https://mlsanity.com/paper/2509.12750/) (2025)
- [VITAL: Vision-Encoder-centered Pre-training for LMMs in Visual Quality Assessment](https://mlsanity.com/paper/2511.17962/) (2025)
- [ELIQ: A Label-Free Framework for Quality Assessment of Evolving AI-Generated Images](https://mlsanity.com/paper/2602.03558/) (2026)
- [EduVQA: Towards Concept-Aware Assessment of Educational AI-Generated Videos](https://mlsanity.com/paper/2603.03066/) (2026)
- [TIQA: Human-Aligned Perceptual Text Quality Assessment in Generated Images](https://mlsanity.com/paper/2603.07119/) (2026)
- [OARS: Process-Aware Online Alignment for Generative Real-World Image Super-Resolution](https://mlsanity.com/paper/2603.12811/) (2026)
- [Towards Characterizing Scientific Image Utility and Upgradability](https://mlsanity.com/paper/2606.03401/) (2026)
- [DriftScope: Measuring The Hidden Effects of Diffusion Model Adaptation](https://mlsanity.com/paper/2607.00183/) (2026)
- [AU-IQA: A Benchmark Dataset for Perceptual Quality Assessment of AI-Enhanced User-Generated Content](https://mlsanity.com/paper/2508.05016/) (2025)
- [DREAM: A Benchmark Study for Deepfake photoREalism AssessMent](https://mlsanity.com/paper/2510.10053/) (2025)
- [Q-REAL: Towards Realism and Plausibility Evaluation for AI-Generated Content](https://mlsanity.com/paper/2511.16908/) (2025)
- [Beyond Pixels: Benchmarking and Reward-Based Assessing Framework for Visual Spatial Aesthetics](https://mlsanity.com/paper/2512.05098/) (2025)
- [Evaluating and Preserving High-level Fidelity in Super-Resolution](https://mlsanity.com/paper/2512.07037/) (2025)
- [Comparison Drives Preference: Reference-Aware Modeling for AI-Generated Video Quality Assessment](https://mlsanity.com/paper/2604.17074/) (2026)
- [Plan-and-Verify Video Reward Reasoning with Spatio-Temporal Scene Graph Grounding](https://mlsanity.com/paper/2606.11838/) (2026)
- [LumaGuide: Distribution Shaping for Training-Free HDR Generation in Diffusion Models](https://mlsanity.com/paper/2607.26237/) (2026)
- [AGIDefect-4K: A Richly Annotated Dataset for AI-Generated Image Defect Detection, Localization and Explanation](https://mlsanity.com/paper/2608.20713/) (2026)

## Abstract (arXiv)

Evaluating text-to-vision content hinges on two crucial aspects: visual quality and alignment. While significant progress has been made in developing objective models to assess these dimensions, the performance of such models heavily relies on the scale and quality of human annotations. According to Scaling Law, increasing the number of human-labeled instances follows a predictable pattern that enhances the performance of evaluation models. Therefore, we introduce a comprehensive dataset designed to Evaluate Visual quality and Alignment Level for text-to-vision content (Q-EVAL-100K), featuring the largest collection of human-labeled Mean Opinion Scores (MOS) for the mentioned two aspects. The Q-EVAL-100K dataset encompasses both text-to-image and text-to-video models, with 960K human annotations specifically focused on visual quality and alignment for 100K instances (60K images and 40K videos). Leveraging this dataset with context prompt, we propose Q-Eval-Score, a unified model capable of evaluating both visual quality and alignment with special improvements for handling long-text prompt alignment. Experimental results indicate that the proposed Q-Eval-Score achieves superior performance on both visual quality and alignment, with strong generalization capabilities across other benchmarks. These findings highlight the significant value of the Q-EVAL-100K dataset. Data and codes will be available at https://github.com/zzc-1998/Q-Eval.