# Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluation

arXiv: https://arxiv.org/abs/2605.28091 · HTML: https://mlsanity.com/paper/2605.28091/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 56 references that are in the map.
- First posted on arXiv 27 May 2026.

## Builds on (most cited first, 30 of 30)

- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2305.01569/) (2023)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis](https://mlsanity.com/paper/2306.09341/) (2023)
- [Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generation](https://mlsanity.com/paper/2410.13848/) (2024)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset](https://mlsanity.com/paper/2505.09568/) (2025)
- [SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning](https://mlsanity.com/paper/2408.05517/) (2024)
- [Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer](https://mlsanity.com/paper/2511.22699/) (2025)
- [T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generation](https://mlsanity.com/paper/2307.06350/) (2023)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2310.18235/) (2023)
- [T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoT](https://mlsanity.com/paper/2505.00703/) (2025)
- [GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generation](https://mlsanity.com/paper/2406.13743/) (2024)
- [HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models](https://mlsanity.com/paper/2304.05390/) (2023)
- [OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation](https://mlsanity.com/paper/2506.07977/) (2025)
- [Classifier-Free Guidance is a Predictor-Corrector](https://mlsanity.com/paper/2408.09000/) (2024)
- [TIIF-Bench: How Does Your T2I Model Follow Your Instructions?](https://mlsanity.com/paper/2506.02161/) (2025)
- [ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty](https://mlsanity.com/paper/2408.14339/) (2024)
- [DesignDiffusion: High-Quality Text-to-Design Image Generation with Diffusion Models](https://mlsanity.com/paper/2503.01645/) (2025)
- [T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation](https://mlsanity.com/paper/2508.17472/) (2025)
- [On the Scalability of Diffusion-based Text-to-Image Generation](https://mlsanity.com/paper/2404.02883/) (2024)
- [R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation](https://mlsanity.com/paper/2505.23493/) (2025)
- [TextCraftor: Your Text Encoder Can be Image Quality Controller](https://mlsanity.com/paper/2403.18978/) (2024)
- [UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation](https://mlsanity.com/paper/2510.18701/) (2025)
- [WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generation](https://mlsanity.com/paper/2505.01490/) (2025)
- [DreamText: High Fidelity Scene Text Synthesis](https://mlsanity.com/paper/2405.14701/) (2024)
- [CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metrics](https://mlsanity.com/paper/2506.08835/) (2025)
- [Scaling Down Text Encoders of Text-to-Image Diffusion Models](https://mlsanity.com/paper/2503.19897/) (2025)
- [Multimodal Language Models as Text-to-Image Model Evaluators](https://mlsanity.com/paper/2505.00759/) (2025)

## Built on by (most cited first, 22 of 22)

- [Sol-Attn: Accelerating Video Generation Inference via On-the-Fly Attention Sparsification](https://mlsanity.com/paper/2607.24027/) (2026)
- [SenseNova-U1.5: Towards Native Unified Visual Intelligence](https://mlsanity.com/paper/2609.11929/) (2026)
- [Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budget](https://mlsanity.com/paper/2607.13125/) (2026)
- [DiffusionBench: On Holistic Evaluation of Diffusion Transformers](https://mlsanity.com/paper/2606.24888/) (2026)
- [Qwen-Image-2.0-RL Technical Report](https://mlsanity.com/paper/2606.27608/) (2026)
- [Text Template Tokens Are Implicit Semantic Registers in Diffusion Transformers](https://mlsanity.com/paper/2607.19139/) (2026)
- [LLaDA-Image: Building Strong Image Generators with Fully Open Training Recipes](https://mlsanity.com/paper/2609.03796/) (2026)
- [Exploring the Performance Frontier of Compact Unified Image Generation Models](https://mlsanity.com/paper/2608.20334/) (2026)
- [WeGenBench: A Multidimensional Diagnostic Benchmark towards Text-to-Image Model Optimization](https://mlsanity.com/paper/2606.20100/) (2026)
- [JoLT: Joint Latent Trajectories for Context-Guided High-Resolution Tiled Generation](https://mlsanity.com/paper/2608.15395/) (2026)
- [Qwen-Image-Flash: Rethinking the Training Recipe for Few-Step Distillation](https://mlsanity.com/paper/2606.03746/) (2026)
- [Scalable Question-Centric Text-to-Image Evaluation: Reliable Ranking, Fine-Grained Diagnosis, and Cost-Aware Routing](https://mlsanity.com/paper/2608.24112/) (2026)
- [DramaChain Bench: An End-to-End Benchmark for Short-Drama Generation](https://mlsanity.com/paper/2609.00646/) (2026)
- [RED: Reconstruction Evolution Dynamics for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2609.36822/) (2026)
- [Think Before You Score: Thinking Reward Model for Visual Generation](https://mlsanity.com/paper/2609.37372/) (2026)
- [CAST: Causal Advantage-Structured Training with Spatially Grounded Compositional Rewards for Diffusion Models](https://mlsanity.com/paper/2609.39441/) (2026)
- [PortraitAes: Intent-Conditioned Structured Portrait Aesthetic Assessment](https://mlsanity.com/paper/2610.05010/) (2026)
- [Two Halves are More than One: Phase-wise Velocity Distillation for Fast and High-Quality Image Generation](https://mlsanity.com/paper/2610.08070/) (2026)
- [UltraText Bench: A Comprehensive Bilingual Benchmark for Evaluating Visual Text Rendering in Image Generation](https://mlsanity.com/paper/2610.09823/) (2026)
- [A Lightweight Velocity Correction Framework for Improving Instruction Following in Few-Step Text-to-Image Generation](https://mlsanity.com/paper/2610.12899/) (2026)
- [Efficient Scene-Text Refinement via Pixel-Space Grouped Image Generation](https://mlsanity.com/paper/2610.18284/) (2026)
- [TextVerifier: Seeing Beyond Recognition to Verify Chinese Text Rendering](https://mlsanity.com/paper/2610.19551/) (2026)

## Abstract (arXiv)

Text-to-Image generation has evolved from basic image synthesis into a frequently used core capability in professional creative workflows, where simple text-image alignment can no longer satisfy users' pressing demands for faithful real-world reconstruction and genuine creative expression. Existing benchmarks, however, remain anchored in these foundational criteria and do not yet capture the nuanced capabilities that matter in authentic artistic practice, making it difficult to reliably distinguish state-of-the-art T2I models. To address the gap, we introduce Qwen-Image-Bench, a creator-centric benchmark co-designed with professional artists and grounded in real-world creation scenarios. Qwen-Image-Bench enriches conventional evaluation with two application-driven dimensions: Real-world Fidelity and Creative Generation. Drawing on the staged reasoning inherent in professional artistic workflows, we organize these five pillars into a top-down hierarchical taxonomy that further decomposes into 23 second-level sub-capabilities and 56 third-level verifiable rubrics. To ensure broad coverage, we curate 1000 stratified prompts with each prompt jointly exercising more than four fine-grained facets across multiple pillars. We train a unified judge model Q-Judger based on Qwen3.6-27B, supervised by 80 professional annotators from global art academies under blind labeling and triple-review protocols, that scores every image across all 56 verifiable facets, producing fine-grained, rubric-grounded, and fully attributable diagnostics rather than a single opaque score. Empirically, Qwen-Image-Bench reliably distinguishes leading T2I models, achieving the greatest separation on the two application-driven dimensions of Real-world Fidelity and Creative Generation where existing benchmarks provide little insight, while also providing a trustworthy optimization signal for production-level T2I development.