mlsanityData to 14 Oct 2026

Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models

arXiv 2605.15792 ↗First posted 15 May 2026cs.CVcited by 1

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

The long-standing goal of multimodal AI is to build unified models in which visual understanding and visual generation mutually enhance one another. Despite recent works such as BAGEL, BLIP3o achieves remarkable progress; In practice, however, this unification remains one-directional: understanding routinely guides generation, yet how and why generation can support understanding is rarely investigated. We revisit this asymmetry and propose Generation-to-Understanding (G2U) synergy, where visual generation becomes an explicit intermediate reasoning step. Our framework enables a model to perform controlled generative acts, such as detail enhancement, context expansion or structural visualisation, to produce self-generated visual thoughts, which are then fed back into the model to refine perception without retraining or external tools. Through a comprehensive evaluation on twelve benchmarks, this reversed information flow consistently improves multimodal understanding. We show that generative fidelity bounds perceptual gain and that distinct families of edit prompts govern transfer efficiency. We further analyse whether models can decide what to imagine. While they can produce plausible edits, these self-generated visual thoughts lack stable task alignment, revealing that current large multimodal models fall short of true self-reflection. This work exposes a missing mechanism in unified cognition and suggests that imagination is not the end of understanding but its beginning.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 47 of its 50 references that are in the map. First posted on arXiv 15 May 2026.

Builds on

47 of 50 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024KiVA: Kid-inspired Visual Analogies for Testing Large Multimodal Models25 Jul
  2. 2024Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models13 Jun
  3. 2024Are We on the Right Way for Evaluating Large Vision-Language Models?29 Mar
  4. 2024CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoning6 Feb

Other references · 43

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2025Qwen2.5-VL Technical Reportcited by 5,587
  3. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,719
  5. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  6. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,141
  7. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  8. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,599
  9. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,239
  10. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  11. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  12. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
31 more
  1. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  2. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  3. 2023HallusionBench: An Advanced Diagnostic Suite for Entangled Language Hallucination and Visual Illusion in Large Vision-Language Modelscited by 583
  4. 2024DeepSeek-VL2: Mixture-of-Experts Vision-Language Models for Advanced Multimodal Understandingcited by 572
  5. 2023V*: Guided Visual Search as a Core Mechanism in Multimodal LLMscited by 482
  6. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  7. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  8. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  9. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  10. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 302
  11. 2025Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learningcited by 278
  12. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  13. 2023Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Visioncited by 249
  14. 2024LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contextscited by 245
  15. 2025Transfer between Modalities with MetaQueriescited by 236
  16. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  17. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 202
  18. 2023VIEScore: Towards Explainable Metrics for Conditional Image Synthesis Evaluationcited by 201
  19. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
  20. 2025OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learningcited by 163
  21. 2025Can MLLMs Reason in Multimodality? EMMA: An Enhanced MultiModal ReAsoning Benchmarkcited by 134
  22. 2025DeepEyesV2: Toward Agentic Multimodal Modelcited by 125
  23. 2024LMFusion: Adapting Pretrained Language Models for Multimodal Generationcited by 120
  24. 2025VTool-R1: VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Usecited by 90
  25. 2025PyVision: Agentic Vision with Dynamic Toolingcited by 79
  26. 2025Visual Agentic Reinforcement Fine-Tuningcited by 55
  27. 2025LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning?cited by 49
  28. 2025DyFo: A Training-Free Dynamic Focus Visual Search for Enhancing LMMs in Fine-Grained Visual Understandingcited by 49
  29. 2025Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Modelscited by 38
  30. 2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?cited by 30
  31. 2025V-Thinker: Interactive Thinking with Imagescited by 26

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

Similar, not cited

Close in wording, no citation either way

  1. 2026UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?
  2. 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?
  3. 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward
  4. 2026UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
  5. 2026Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Models
  6. 2026CLEAR: Unlocking Generative Potential for Degraded Image Understanding in Unified Multimodal Models
  7. 2025Thinking with Generated Images
  8. 2025Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark