# RealUnify: Do Unified Models Truly Benefit from Unification? A Comprehensive Benchmark

arXiv: https://arxiv.org/abs/2509.24897 · HTML: https://mlsanity.com/paper/2509.24897/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 47 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Builds on (most cited first, 30 of 39)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [Qwen-Image Technical Report](https://mlsanity.com/paper/2508.02324/) (2025)
- [FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Space](https://mlsanity.com/paper/2506.15742/) (2025)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [Chameleon: Mixed-Modal Early-Fusion Foundation Models](https://mlsanity.com/paper/2405.09818/) (2024)
- [Emerging Properties in Unified Multimodal Pretraining](https://mlsanity.com/paper/2505.14683/) (2025)
- [Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling](https://mlsanity.com/paper/2501.17811/) (2025)
- [Emu3: Next-Token Prediction is All You Need](https://mlsanity.com/paper/2409.18869/) (2024)
- [BLINK: Multimodal Large Language Models Can See but Not Perceive](https://mlsanity.com/paper/2404.12390/) (2024)
- [BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Dataset](https://mlsanity.com/paper/2505.09568/) (2025)
- [Show-o2: Improved Native Unified Multimodal Models](https://mlsanity.com/paper/2506.15564/) (2025)
- [LogicVista: Multimodal LLM Logical Reasoning Benchmark in Visual Contexts](https://mlsanity.com/paper/2407.04973/) (2024)
- [WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generation](https://mlsanity.com/paper/2503.07265/) (2025)
- [Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontiers](https://mlsanity.com/paper/2506.23918/) (2025)
- [Thyme: Think Beyond Images](https://mlsanity.com/paper/2508.11630/) (2025)
- [Divide, Conquer and Combine: A Training-Free Framework for High-Resolution Image Perception in Multimodal Large Language Models](https://mlsanity.com/paper/2408.15556/) (2024)
- [Can We Generate Images with CoT? Let's Verify and Reinforce Image Generation Step by Step](https://mlsanity.com/paper/2501.13926/) (2025)
- [Machine Mental Imagery: Empower Multimodal Reasoning with Latent Visual Tokens](https://mlsanity.com/paper/2506.17218/) (2025)
- [Ovis-U1 Technical Report](https://mlsanity.com/paper/2506.23044/) (2025)
- [Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models](https://mlsanity.com/paper/2403.12966/) (2024)
- [Liquid: Language Models are Scalable and Unified Multi-modal Generators](https://mlsanity.com/paper/2412.04332/) (2024)
- [ILLUME+: Illuminating Unified MLLM with Dual Visual Tokenization and Diffusion Refinement](https://mlsanity.com/paper/2504.01934/) (2025)
- [Unified Autoregressive Visual Generation and Understanding with Continuous Tokens](https://mlsanity.com/paper/2503.13436/) (2025)
- [OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generation](https://mlsanity.com/paper/2509.03498/) (2025)
- [T2I-ReasonBench: Benchmarking Reasoning-Informed Text-to-Image Generation](https://mlsanity.com/paper/2508.17472/) (2025)
- [ImageGen-CoT: Enhancing Text-to-Image In-context Learning with Chain-of-Thought Reasoning](https://mlsanity.com/paper/2503.19312/) (2025)
- [Easier Painting Than Thinking: Can Text-to-Image Models Set the Stage, but Not Direct the Play?](https://mlsanity.com/paper/2509.03516/) (2025)

## Built on by (most cited first, 30 of 33)

- [Monet: Reasoning in Latent Visual Space Beyond Images and Language](https://mlsanity.com/paper/2511.21395/) (2025)
- [SimScale: Learning to Drive via Real-World Simulation at Scale](https://mlsanity.com/paper/2511.23369/) (2025)
- [SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture](https://mlsanity.com/paper/2605.12500/) (2026)
- [Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Image](https://mlsanity.com/paper/2512.16899/) (2025)
- [Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models](https://mlsanity.com/paper/2601.19834/) (2026)
- [Quantifying the Gap between Understanding and Generation within Unified Multimodal Models](https://mlsanity.com/paper/2602.02140/) (2026)
- [GENIUS: Generative Fluid Intelligence Evaluation Suite](https://mlsanity.com/paper/2602.11144/) (2026)
- [Imagination Helps Visual Reasoning, But Not Yet in Latent Space](https://mlsanity.com/paper/2602.22766/) (2026)
- [VTC-Bench: Evaluating Agentic Multimodal Models via Compositional Visual Tool Chaining](https://mlsanity.com/paper/2603.15030/) (2026)
- [MammothModa2: A Unified AR-Diffusion Framework for Multimodal Understanding and Generation](https://mlsanity.com/paper/2511.18262/) (2025)
- [UEval: A Benchmark for Unified Multimodal Generation](https://mlsanity.com/paper/2601.22155/) (2026)
- [BrowseComp-V3: A Visual, Vertical, and Verifiable Benchmark for Multimodal Browsing Agents](https://mlsanity.com/paper/2602.12876/) (2026)
- [SenseNova-U1.5: Towards Native Unified Visual Intelligence](https://mlsanity.com/paper/2609.11929/) (2026)
- [VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning](https://mlsanity.com/paper/2608.26105/) (2026)
- [FysicsWorld: A Unified Full-Modality Benchmark for Any-to-Any Understanding, Generation, and Reasoning](https://mlsanity.com/paper/2512.12756/) (2025)
- [Self-Improving World Modelling with Latent Actions](https://mlsanity.com/paper/2602.06130/) (2026)
- [Agentic-MME: What Agentic Capability Really Brings to Multimodal Intelligence?](https://mlsanity.com/paper/2604.03016/) (2026)
- [Beyond Accuracy: Benchmarking Cross-Task Consistency in Unified Multimodal Models](https://mlsanity.com/paper/2604.25072/) (2026)
- [Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks](https://mlsanity.com/paper/2604.19697/) (2026)
- [Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenization](https://mlsanity.com/paper/2605.10780/) (2026)
- [Towards Physics of Multimodal Pretraining: Knowledge Flow, Modality Synergy, Early Unification, and Recipes](https://mlsanity.com/paper/2608.05000/) (2026)
- [TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training](https://mlsanity.com/paper/2604.10784/) (2026)
- [Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Models](https://mlsanity.com/paper/2608.09666/) (2026)
- [AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Model](https://mlsanity.com/paper/2511.22663/) (2025)
- [Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?](https://mlsanity.com/paper/2602.23711/) (2026)
- [Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models](https://mlsanity.com/paper/2604.10949/) (2026)
- [LatentUMM: Dual Latent Alignment for Unified Multimodal Models](https://mlsanity.com/paper/2605.17766/) (2026)
- [Benchmarking and Evolving Reason-Reflect-Rectify for Reflective Visual Generation](https://mlsanity.com/paper/2605.19639/) (2026)
- [PixelUMM: Encoder-Free Unified Image and Video Understanding and Generation](https://mlsanity.com/paper/2609.38597/) (2026)
- [FRBNet: Revisiting Low-Light Vision through Frequency-Domain Radial Basis Network](https://mlsanity.com/paper/2510.23444/) (2025)

## Abstract (arXiv)

The integration of visual understanding and generation into unified multimodal models represents a significant stride toward general-purpose AI. However, a fundamental question remains unanswered by existing benchmarks: does this architectural unification actually enable synergetic interaction between the constituent capabilities? Existing evaluation paradigms, which primarily assess understanding and generation in isolation, are insufficient for determining whether a unified model can leverage its understanding to enhance its generation, or use generative simulation to facilitate deeper comprehension. To address this critical gap, we introduce RealUnify, a benchmark specifically designed to evaluate bidirectional capability synergy. RealUnify comprises 1,000 meticulously human-annotated instances spanning 10 categories and 32 subtasks. It is structured around two core axes: 1) Understanding Enhances Generation, which requires reasoning (e.g., commonsense, logic) to guide image generation, and 2) Generation Enhances Understanding, which necessitates mental simulation or reconstruction (e.g., of transformed or disordered visual inputs) to solve reasoning tasks. A key contribution is our dual-evaluation protocol, which combines direct end-to-end assessment with a diagnostic stepwise evaluation that decomposes tasks into distinct understanding and generation phases. This protocol allows us to precisely discern whether performance bottlenecks stem from deficiencies in core abilities or from a failure to integrate them. Through large-scale evaluations of 12 leading unified models and 6 specialized baselines, we find that current unified models still struggle to achieve effective synergy, indicating that architectural unification alone is insufficient. These results highlight the need for new training strategies and inductive biases to fully unlock the potential of unified modeling.