Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models
On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning
Abstract (arXiv)
The long-standing goal of multimodal AI is to build unified models in which visual understanding and visual generation mutually enhance one another. Despite recent works such as BAGEL, BLIP3o achieves remarkable progress; In practice, however, this unification remains one-directional: understanding routinely guides generation, yet how and why generation can support understanding is rarely investigated. We revisit this asymmetry and propose Generation-to-Understanding (G2U) synergy, where visual generation becomes an explicit intermediate reasoning step. Our framework enables a model to perform controlled generative acts, such as detail enhancement, context expansion or structural visualisation, to produce self-generated visual thoughts, which are then fed back into the model to refine perception without retraining or external tools. Through a comprehensive evaluation on twelve benchmarks, this reversed information flow consistently improves multimodal understanding. We show that generative fidelity bounds perceptual gain and that distinct families of edit prompts govern transfer efficiency. We further analyse whether models can decide what to imagine. While they can produce plausible edits, these self-generated visual thoughts lack stable task alignment, revealing that current large multimodal models fall short of true self-reflection. This work exposes a missing mechanism in unified cognition and suggests that imagination is not the end of understanding but its beginning.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 50 references that are in the map. First posted on arXiv 15 May 2026.
Builds on
47 of 50 references.
Closest ancestors · 4
Same direction, newest first
- 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models25 Jul
- 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
- 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar
- 2024CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoning6 Feb
Other references · 43
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
31 more
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
- 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
- 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
- 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
- 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 249
- 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 245
- 2025Transfer between Modalities with MetaQueriescited by 236
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
- 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
- 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
- 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
- 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 134
- 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
- 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
- 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Usecited by 90
- 2025PyVision: Agentic Vision with Dynamic Toolingcited by 79
- 2025Visual Agentic Reinforcement Fine-Tuningcited by 55
- 2025LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?cited by 49
- 2025DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understandingcited by 49
- 2025Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Modelscited by 38
- 2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?cited by 30
- 2025V-Thinker: Interactive Thinking with Imagescited by 26
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
- 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward
- 2026UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
- 2026Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models
- 2026CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Models
- 2025Thinking with Generated Images
- 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark