mlsanityData to 14 Oct 2026

Beyond Words and Pixels: A Benchmark for Implicit World Knowledge Reasoning in Generative Models

arXiv 2511.18271 ↗First posted 23 Nov 2025cs.CV cs.AIcited by 4

Not on the map. Why?

Abstract (arXiv)

Text-to-image (T2I) models today are capable of producing photorealistic, instruction-following images, yet they still frequently fail on prompts that require implicit world knowledge. Existing evaluation protocols either emphasize compositional alignment or rely on single-round VQA-based scoring, leaving critical dimensions such as knowledge grounding, multi-physics interactions, and auditable evidence-substantially undertested. To address these limitations, we introduce PicWorld, the first comprehensive benchmark that assesses the grasp of implicit world knowledge and physical causal reasoning of T2I models. This benchmark consists of 1,100 prompts across three core categories. To facilitate fine-grained evaluation, we propose PW-Agent, an evidence-grounded multi-agent evaluator to hierarchically assess images on their physical realism and logical consistency by decomposing prompts into verifiable visual evidence. We conduct a thorough analysis of 17 mainstream T2I models on PicWorld, illustrating that they universally exhibit a fundamental limitation in their capacity for implicit world knowledge and physical causal reasoning to varying degrees. The findings highlight the need for reasoning-aware, knowledge-integrative architectures in future T2I systems.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 36 of its 39 references that are in the map. First posted on arXiv 23 Nov 2025.

Builds on

36 of 39 references.

References · 36

  1. 2025Qwen2.5-VL Technical Reportcited by 5,587
  2. 2024GPT-4o System Cardcited by 4,588
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  6. 2024A Survey on LLM-as-a-Judgecited by 1,423
  7. 2025Qwen-Image Technical Reportcited by 1,044
  8. 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,023
  9. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  10. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  11. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  12. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
24 more
  1. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 474
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  3. 2024Hallucination of Multimodal Large Language Models: A Surveycited by 388
  4. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  5. 2024SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformerscited by 340
  6. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generationcited by 329
  7. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  8. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 310
  9. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  10. 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 249
  11. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  12. 2025HiDream-I1: A High-Efficient Image Generative Foundation Model with Sparse Diffusion Transformercited by 141
  13. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 96
  14. 2024Can You Trust LLM Judgments? Reliability of LLM-as-a-Judgecited by 56
  15. 2025BLIP3o-NEXT: Next Frontier of Native Image Generationcited by 54
  16. 2024Commonsense-T2I Challenge: Can Text-to-Image Generation Models Understand Commonsense?cited by 52
  17. 2025R2I-Bench: Benchmarking Reasoning-Driven Text-to-Image Generationcited by 32
  18. 2025WorldGenBench: A World-Knowledge-Integrated Benchmark for Reasoning-Driven Text-to-Image Generationcited by 20
  19. 2025Failure Makes the Agent Stronger: Enhancing Accuracy through Structured Reflection for Reliable Tool Interactionscited by 12
  20. 2024The Instinctive Bias: Spurious Images lead to Illusion in MLLMscited by 11
  21. 2025MR. Judge: Multimodal Reasoner as a Judgecited by 7
  22. 2024ProReason: Multi-Modal Proactive Reasoning with Decoupled Eyesight and Wisdomcited by 5
  23. 2025PositionIC: Unified Position and Identity Consistency for Image Customizationcited by 4
  24. 2024High-Resolution Image Synthesis via Next-Token Predictioncited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2026Are Text-to-Image Models Inductivist Turkeys? A Counterfactual Benchmark for Causal Reasoning
  2. 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights
  3. 2025MMGR: Multi-Modal Generative Reasoning Benchmark and Evaluation
  4. 2026UniReason 1.0: A Unified Reasoning Framework for World Knowledge Aligned Image Generation and Editing
  5. 2026ExpertVerse: A General-Purpose Benchmark for Expert-Level Reasoning in Knowledge-Intensive Visual Synthesis
  6. 2025MMMG: A Massive, Multidisciplinary, Multi-Tier Generation Benchmark for Text-to-Image Reasoning
  7. 2026Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators
  8. 2025Synthetic Perception: Can Generated Images Unlock Latent Visual Prior for Text-Centric Reasoning?