mlsanityData to 14 Oct 2026

OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation

arXiv 2506.07977 ↗First posted 9 Jun 2025cs.CVcited by 87

Not on the map. Why?

Cited by 21 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Text-to-image (T2I) models have garnered significant attention for generating high-quality images aligned with text prompts. However, rapid T2I model advancements reveal limitations in early benchmarks, lacking comprehensive evaluations, for example, the evaluation on reasoning, text rendering and style. Notably, recent state-of-the-art models, with their rich knowledge modeling capabilities, show promising results on the image generation problems requiring strong reasoning ability, yet existing evaluation systems have not adequately addressed this frontier. To systematically address these gaps, we introduce OneIG-Bench, a meticulously designed comprehensive benchmark framework for fine-grained evaluation of T2I models across multiple dimensions, including prompt-image alignment, text rendering precision, reasoning-generated content, stylization, and diversity. By structuring the evaluation, this benchmark enables in-depth analysis of model performance, helping researchers and practitioners pinpoint strengths and bottlenecks in the full pipeline of image generation. Specifically, OneIG-Bench enables flexible evaluation by allowing users to focus on a particular evaluation subset. Instead of generating images for the entire set of prompts, users can generate images only for the prompts associated with the selected dimension and complete the corresponding evaluation accordingly. Our codebase and dataset are now publicly available to facilitate reproducible evaluation studies and cross-model comparisons within the T2I research community.

Key facts. Cited by 87 papers in the map (data through 14 Oct 2026). Builds on 51 of its 74 references that are in the map. First posted on arXiv 9 Jun 2025.

Builds on

51 of 74 references.

References · 51

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2022Generative Adversarial Networkscited by 5,627
  9. 2025Qwen2.5-VL Technical Reportcited by 5,587
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  11. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  12. 2021Zero-Shot Text-to-Image Generationcited by 4,288
38 more
  1. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  2. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  3. 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
  4. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  7. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,385
  8. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  9. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 977
  10. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  11. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  12. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  13. 2023Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesiscited by 890
  14. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  15. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  16. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 777
  17. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  18. 2024SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformerscited by 737
  19. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Modelscited by 702
  20. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  21. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 428
  22. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesiscited by 388
  23. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  24. 2023T2I-CompBench++: An Enhanced and Comprehensive Benchmark for Compositional Text-to-image Generationcited by 260
  25. 2023StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesiscited by 212
  26. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  27. 2021Mask2Former for Video Instance Segmentationcited by 171
  28. 2023Davidsonian Scene Graph: Improving Reliability in Fine-grained Evaluation for Text-to-Image Generationcited by 171
  29. 2025Seedream 3.0 Technical Reportcited by 154
  30. 2024GenAI-Bench: Evaluating and Improving Compositional Text-to-Visual Generationcited by 135
  31. 2024Measuring Style Similarity in Diffusion Modelscited by 109
  32. 2023HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Modelscited by 95
  33. 2024CSGO: Content-Style Composition in Text-to-Image Generationcited by 78
  34. 2025PHYBench: Holistic Evaluation of Physical Perception and Reasoning in Large Language Modelscited by 61
  35. 2024Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?cited by 52
  36. 2024Revisiting Text-to-Image Evaluation with Gecko: On Metrics, Prompts, and Human Ratingscited by 44
  37. 2024EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluationcited by 39
  38. 2025Distilling Diversity and Control in Diffusion Modelscited by 22

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 21; 2026: 66.

87 papers.

Sort:

By area:

  1. 2025Qwen-Image Technical Reportcited by 1,044
  2. 2025Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformercited by 277
  3. 2025OmniGen2: Towards Instruction-Aligned Multimodal Generationcited by 253
  4. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  5. 2025Emu3.5: Native Multimodal Models are World Learnerscited by 132
  6. 2025X-Omni: Reinforcement Learning Makes Discrete Autoregressive Image Generative Models Great Againcited by 112
  7. 2025Lumina-DiMOO: An Omni Diffusion Large Language Model for Multi-Modal Generation and Understandingcited by 88
  8. 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 57
  9. 2025GenEval 2: Addressing Benchmark Drift in Text-to-Image Evaluationcited by 52
  10. 2025Interleaving Reasoning for Better Text-to-Image Generationcited by 41
  11. 2025Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?cited by 34
  12. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
38 more
  1. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 32
  2. 2025pi-Flow: Policy-Based Few-Step Generation via Imitation Distillationcited by 29
  3. 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 28
  4. 2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 26
  5. 2025GenExam: A Multidisciplinary Text-to-Image Examcited by 22
  6. 2026Qwen-Image-Bench: From Generation to Creation in Text-to-Image Evaluationcited by 22
  7. 2026GEMS: Agent-Native Multimodal Generation with Memory and Skillscited by 21
  8. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Imagecited by 20
  9. 2026IGenBench: Benchmarking the Reliability of Text-to-Infographic Generationcited by 17
  10. 2026Everything in Its Place: Benchmarking Spatial Intelligence of Text-to-Image Modelscited by 16
  11. 2026TextPecker: Rewarding Structural Anomaly Quantification for Enhancing Visual Text Renderingcited by 15
  12. 2026PresentBench: A Fine-Grained Rubric-Based Benchmark for Slide Generationcited by 14
  13. 2026LLaDA2.0-Uni: Unifying Multimodal Understanding and Generation with Diffusion Large Language Modelcited by 13
  14. 2026JoyAI-Image: Awaking Spatial Intelligence in Unified Multimodal Understanding and Generationcited by 12
  15. 2026BizGenEval: A Systematic Benchmark for Commercial Visual Content Generationcited by 11
  16. 2026Parallel Decoding Distillation for Fast Image and Video Generationcited by 10
  17. 2025OCRGenBench: A Comprehensive Benchmark for Evaluating OCR Generative Capabilitiescited by 9
  18. 2025Ovis-Image Technical Reportcited by 8
  19. 2026UniT: Unified Multimodal Chain-of-Thought Test-time Scalingcited by 8
  20. 2025Pluggable Pruning with Contiguous Layer Distillation for Diffusion Transformerscited by 7
  21. 2026UReason: Benchmarking Reasoning-to-Generation Alignment in Unified Multimodal Modelscited by 7
  22. 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 7
  23. 2026M3: High-fidelity Text-to-Image Generation via Multi-Modal, Multi-Agent and Multi-Round Visual Reasoningcited by 6
  24. 2026CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generationcited by 6
  25. 2026GlyphPrinter: Region-Grouped Direct Preference Optimization for Glyph-Accurate Visual Text Renderingcited by 6
  26. 2026Lens: Rethinking Training Efficiency for Foundational Text-to-Image Modelscited by 6
  27. 2026ERNIE-Image Technical Reportcited by 6
  28. 2026COMFYCLAW: Self-Evolving Skill Harnesses for Image Generation Workflowscited by 6
  29. 2026Boogu-Image-0.1: Boosting Open Agentic Multimodal Generation via Understanding under a Minimal Budgetcited by 6
  30. 2026An Empirical Study of Training Pixel-Space Text-to-Image Diffusion Modelscited by 6
  31. 2026MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Modelscited by 5
  32. 2026Finite Difference Flow Optimization for RL Post-Training of Text-to-Image Modelscited by 5
  33. 2026Nucleus-Image: Sparse MoE for Image Generationcited by 5
  34. 2026Visual Generation in the New Era: An Evolution from Atomic Mapping to Agentic World Modelingcited by 5
  35. 2026GenRouter: Unified Workflow Routing for Agentic Image Generationcited by 5
  36. 2026BitDance: Scaling Autoregressive Generative Models with Binary Tokenscited by 4
  37. 2026Guidance Matters: Rethinking the Evaluation Pitfall for Text-to-Image Generationcited by 4
  38. 2026ServImage: An Image Generation and Editing Benchmark from Real-world Commercial Imaging Servicescited by 4

50 most cited of 87 All 87 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025UniGenBench++: A Unified Semantic Evaluation Benchmark for Text-to-Image Generation
  2. 2025Draw ALL Your Imagine: A Holistic Benchmark and Agent Framework for Complex Instruction-based Image Generation
  3. 2025R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generation
  4. 2025TIIF-Bench: How Does Your T2I Model Follow Your Instructions?
  5. 2025TIT-Score: Evaluating Long-Prompt Based Text-to-Image Alignment via Text-to-Image-to-Text Consistency
  6. 2025M3T2IBench: A Large-Scale Multi-Category, Multi-Instance, Multi-Relation Text-to-Image Benchmark
  7. 2025Align Beyond Prompts: Evaluating World Knowledge Alignment in Text-to-Image Generation
  8. 2022Human Evaluation of Text-to-Image Models on a Multi-Task Benchmark