Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models
On the map: World models → World models for language agents
also in 1 more
Abstract (arXiv)
Humans construct internal world models and reason by manipulating the concepts within these models. Recent advances in AI, particularly chain-of-thought (CoT) reasoning, approximate such human cognitive abilities, where world models are believed to be embedded within large language models. Expert-level performance in formal and abstract domains such as mathematics and programming has been achieved in current systems by relying predominantly on verbal reasoning. However, they still lag far behind humans in domains like physical and spatial intelligence, which require richer representations and prior knowledge. The emergence of unified multimodal models (UMMs) capable of both verbal and visual generation has therefore sparked interest in more human-like reasoning grounded in complementary multimodal pathways, though their benefits remain unclear. From a world-model perspective, this paper presents the first principled study of when and how visual generation benefits reasoning. Our key position is the visual superiority hypothesis: for certain tasks--particularly those grounded in the physical world--visual generation more naturally serves as world models, whereas purely verbal world models encounter bottlenecks arising from representational limitations or insufficient prior knowledge. Theoretically, we formalize internal world modeling as a core component of CoT reasoning and analyze distinctions among different forms of world models. Empirically, we identify tasks that necessitate interleaved visual-verbal CoT reasoning, constructing a new evaluation suite, VisWorld-Eval. Controlled experiments on a state-of-the-art UMM show that interleaved CoT significantly outperforms purely verbal CoT on tasks that favor visual world modeling, but offers no clear advantage otherwise. Together, this work clarifies the potential of multimodal world modeling for more powerful, human-like multimodal AI.
Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Placed in World models for language agents, one of 11 directions in World models. Builds on 58 of its 79 references that are in the map. First posted on arXiv 27 Jan 2026.
Builds on
58 of 79 references.
Closest ancestors · 3
Same direction, newest first
- 2025VAGEN: Reinforcing World Model Reasoning for Multi-Turn VLM Agents19 Oct
- 2025CWM: An Open-Weights LLM for Research on Code Generation with World Models30 Sep
- 2025Cosmos World Foundation Model Platform for Physical AI7 Jan
Other references · 55
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2024GPT-4o System Cardcited by 4,588
- 2025Qwen3-VL Technical Reportcited by 2,793
- 2018World Modelscited by 1,513
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2019Mastering Atari, Go, Chess and Shogi by Planning with a Learned Modelcited by 1,353
- 2023Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecturecited by 1,039
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planningcited by 809
38 more
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 661
- 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 483
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
- 2025CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Modelscited by 451
- 2022Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Taskcited by 431
- 2022Temporal Difference Learning for Model Predictive Controlcited by 397
- 2024The Platonic Representation Hypothesiscited by 382
- 2025Seed1.5-VL Technical Reportcited by 373
- 2024Diffusion for World Modeling: Visual Details Matter in Ataricited by 300
- 2025Seedream 4.0: Toward Next-generation Multimodal Image Generationcited by 249
- 2025Transfer between Modalities with MetaQueriescited by 236
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
- 2025MMSI-Bench: A Benchmark for Multi-Image Spatial Intelligencecited by 183
- 2021Embodied Intelligence via Learning and Evolutioncited by 163
- 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 163
- 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 146
- 2025Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Stepcited by 131
- 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 99
- 2025MindCube: Spatial Mental Modeling from Limited Viewscited by 95
- 2025Visual Spatial Tuningcited by 89
- 2025Visual Planning: Let's Think Only with Imagescited by 85
- 2025Agent Learning via Early Experiencecited by 85
- 2025Zebra-CoT: A Dataset for Interleaved Vision Language Reasoningcited by 84
- 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 56
- 2024Can Language Models Serve as Text-Based World Simulators?cited by 52
- 2025Reflective Planning: Vision-Language Models for Multi-Stage Long-Horizon Robotic Manipulationcited by 42
- 2025RLVR-World: Training World Models with Reinforcement Learningcited by 40
- 2023Visual cognition in multimodal large language modelscited by 38
- 2025Planning with Reasoning using Vision Language World Modelcited by 36
- 2025MathCanvas: Intrinsic Visual Chain-of-Thought for Multimodal Mathematical Reasoningcited by 35
- 2025RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmarkcited by 33
- 2025Game-RL: Synthesizing Multimodal Verifiable Game Data to Boost VLMs' General Reasoningcited by 32
- 2023A Configurable Library for Generating and Manipulating Maze Datasetscited by 31
- 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmarkcited by 29
50 most cited of 55 All 55 (JSON)
Built on by
Citing papers per year: 2026: 19.
19 papers.
Sort:
By area:
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 14
- 2026Human Cognition in Machines: A Unified Perspective of World Modelscited by 5
- 2026Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulatorscited by 5
- 2026Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Processcited by 5
- 2026Scaling GUI Agents with Visual State Transitionscited by 4
- 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
- 2026Steering Visual Generation in Unified Multimodal Models with Understanding Supervisioncited by 3
- 2026Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoningcited by 3
- 2026Advances in GRPO for Generation Models: A Surveycited by 2
- 2026How Far Are Video Models from True Multimodal Reasoning?cited by 2
- 2026World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoningcited by 2
- 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Textcited by 2
7 more
- 2026Do multimodal models imagine electric sheep?cited by 1
- 2026Hierarchical Denoising For Multi-Step Visual Reasoningcited by 1
- 2026StateSight: Benchmarking Latent Spatial-State Reconstruction in Vision-Language Modelscited by 1
- 2026UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generationcited by 0
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoningcited by 0
- 2026Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcementcited by 0
- 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoningcited by 0
- 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning28 Sep 2026
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning27 Aug 2026
- 2026StateSight: Benchmarking Latent Spatial-State Reconstruction in Vision-Language Models15 Aug 2026
- 2026Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoning10 Aug 2026
- 2026Scaling GUI Agents with Visual State Transitions27 Jul 2026
- 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text23 Jul 2026
- 2026Hierarchical Denoising For Multi-Step Visual Reasoning16 Jul 2026
- 2026Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process4 Jul 2026
- 2026Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement11 Jun 2026
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning11 Jun 2026
- 2026Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators4 Jun 2026
- 2026UniCanvas: A Diffusion-base Unified Model for Text-in-Image Joint Generation2 Jun 2026
7 more
- 2026World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning2 Jun 2026
- 2026Do multimodal models imagine electric sheep?10 May 2026
- 2026Steering Visual Generation in Unified Multimodal Models with Understanding Supervision7 May 2026
- 2026How Far Are Video Models from True Multimodal Reasoning?21 Apr 2026
- 2026Human Cognition in Machines: A Unified Perspective of World Models17 Apr 2026
- 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Models6 Apr 2026
- 2026Advances in GRPO for Generation Models: A Survey21 Feb 2026
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 14
- 2026Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulatorscited by 5
- 2026Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Processcited by 5
- 2026Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoningcited by 3
- 2026Advances in GRPO for Generation Models: A Surveycited by 2
- 2026How Far Are Video Models from True Multimodal Reasoning?cited by 2
- 2026Hierarchical Denoising For Multi-Step Visual Reasoningcited by 1
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoningcited by 0
- 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning27 Aug 2026
- 2026Thinking With Tools, Not With Pixels: Tool Calls as Text Scaffolds for Visual Reasoning10 Aug 2026
- 2026Hierarchical Denoising For Multi-Step Visual Reasoning16 Jul 2026
- 2026Bridging Interleaved Multi-Modal Reasoning as a Unified Decision Process4 Jul 2026
- 2026Perceive, Interact, Reason: Building Tool-Augmented Visual Agents for Spatial Reasoning11 Jun 2026
- 2026Thinking with Imagination: Agentic Visual Spatial Reasoning with World Simulators4 Jun 2026
- 2026How Far Are Video Models from True Multimodal Reasoning?21 Apr 2026
- 2026Advances in GRPO for Generation Models: A Survey21 Feb 2026
- 2026Human Cognition in Machines: A Unified Perspective of World Modelscited by 5
- 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Modelscited by 3
- 2026World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoningcited by 2
- 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoningcited by 0
- 2026WM-VLM: Probing Internal World Models for Interleaved Visual-Textual Reasoning28 Sep 2026
- 2026World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning2 Jun 2026
- 2026Human Cognition in Machines: A Unified Perspective of World Models17 Apr 2026
- 2026OpenWorldLib: A Unified Codebase and Definition of Advanced World Models6 Apr 2026
- 2026Scaling GUI Agents with Visual State Transitionscited by 4
- 2026Scaling GUI Agents with Visual State Transitions27 Jul 2026
- 2026Steering Visual Generation in Unified Multimodal Models with Understanding Supervisioncited by 3
- 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Textcited by 2
- 2026Do multimodal models imagine electric sheep?cited by 1
- 2026Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcementcited by 0
- 2026Show, Don't Tell: Evaluating Spatial Cognition in Generative Pixels Rather Than LLM Text23 Jul 2026
- 2026Bridging Modal Isolation in Interleaved Thinking: Supervising Modality Transitions via Stepwise Reinforcement11 Jun 2026
- 2026Do multimodal models imagine electric sheep?10 May 2026
- 2026Steering Visual Generation in Unified Multimodal Models with Understanding Supervision7 May 2026
Similar, not cited
Close in wording, no citation either way
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers
- 2025Thinking with Generated Images
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
- 2025From Perception to Cognition: A Survey of Vision-Language Interactive Reasoning in Multimodal Large Language Models
- 2025Can World Simulators Reason? Gen-ViRe: A Generative Visual Reasoning Benchmark
- 2025Spatial Reasoning in Multimodal Large Language Models: A Survey of Tasks, Benchmarks and Methods
- 2026Look Light, Think Heavy: What Multimodal Chain-of-Thought Reasoning Can and Cannot Do
- 2026UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?