Uni-MMMU: A Massive Multi-discipline Multimodal Unified Benchmark
Not on the map. Why?
Abstract (arXiv)
Unified multimodal models aim to jointly enable visual understanding and generation, yet current benchmarks rarely examine their true integration. Existing evaluations either treat the two abilities in isolation or overlook tasks that inherently couple them. To address this gap, we present Uni-MMMU, a comprehensive and discipline-aware benchmark that systematically unfolds the bidirectional synergy between generation and understanding across eight reasoning-centric domains, including science, coding, mathematics, and puzzles. Each task is bidirectionally coupled, demanding models to (i) leverage conceptual understanding to guide precise visual synthesis, or (ii) utilize generation as a cognitive scaffold for analytical reasoning. Uni-MMMU incorporates verifiable intermediate reasoning steps, unique ground truths, and a reproducible scoring protocol for both textual and visual outputs. Through extensive evaluation of state-of-the-art unified, generation-only, and understanding-only models, we reveal substantial performance disparities and cross-modal dependencies, offering new insights into when and how these abilities reinforce one another, and establishing a reliable foundation for advancing unified models.
Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 16 of its 35 references that are in the map. First posted on arXiv 15 Oct 2025.
Builds on
16 of 35 references.
References · 16
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,080
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2025VBench-2.0: Advancing Video Generation Benchmark Suite for Intrinsic Faithfulnesscited by 315
- 2025Thinking with Images for Multimodal Reasoning: Foundations, Methods, and Future Frontierscited by 199
- 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 99
- 2025ShareGPT-4o-Image: Aligning Multimodal Models with GPT-4o-Level Image Generationcited by 97
- 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 45
- 2024Evaluation Agent: Efficient and Promptable Evaluation Framework for Visual Generative Modelscited by 41
- 2025Skywork UniPic 2.0: Building Kontext Model with Online RL for Unified Multimodal Modelcited by 41
4 more
- 2024OpenING: A Comprehensive Benchmark for Judging Open-ended Interleaved Image-Text Generationcited by 31
- 2025MME-Unify: A Comprehensive Benchmark for Unified Multimodal Understanding and Generation Modelscited by 24
- 2024Interleaved Scene Graphs for Interleaved Text-and-Image Generation Assessmentcited by 21
- 2025The Telephone Game: Evaluating Semantic Drift in Unified Modelscited by 4
Built on by
Citing papers per year: 2025: 3; 2026: 26.
29 papers.
Sort:
By area:
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 32
- 2026MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agentscited by 22
- 2026Visual Generation Unlocks Human-Like Reasoning through Multimodal World Modelscited by 19
- 2026Demystifying Video Reasoningcited by 14
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 12
- 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 10
- 2026UEval: A Benchmark for Unified Multimodal Generationcited by 9
- 2026Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomaliescited by 8
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligencecited by 7
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoningcited by 6
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
- 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insightscited by 2
17 more
- 2026TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-trainingcited by 2
- 2026Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Modelscited by 2
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?cited by 2
- 2025AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Modelcited by 1
- 2026EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understandingcited by 1
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Modelscited by 1
- 2026LatentUMM: Dual Latent Alignment for Unified Multimodal Modelscited by 1
- 2026DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Modelscited by 1
- 2026Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generationcited by 1
- 2026The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math?cited by 0
- 2026SketchVLM: Vision language models can annotate images to explain thoughts and guide userscited by 0
- 2026The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Spacecited by 0
- 2026InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Rewardcited by 0
- 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generationcited by 0
- 2026UniVR: Thinking in Visual Space for Unified Visual Reasoningcited by 0
- 2026V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learningcited by 0
- 2026What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generationcited by 0
- 2026What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation29 Sep 2026
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligence10 Sep 2026
- 2026Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation2 Sep 2026
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning26 Aug 2026
- 2026V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning26 Aug 2026
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?23 Aug 2026
- 2026Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Models10 Aug 2026
- 2026UniVR: Thinking in Visual Space for Unified Visual Reasoning14 Jul 2026
- 2026DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models9 Jul 2026
- 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generation8 Jun 2026
- 2026InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward26 May 2026
- 2026LatentUMM: Dual Latent Alignment for Unified Multimodal Models18 May 2026
17 more
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture12 May 2026
- 2026The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space11 May 2026
- 2026SketchVLM: Vision language models can annotate images to explain thoughts and guide users23 Apr 2026
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks21 Apr 2026
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models13 Apr 2026
- 2026TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training12 Apr 2026
- 2026EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding18 Mar 2026
- 2026Demystifying Video Reasoning17 Mar 2026
- 2026The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math?11 Feb 2026
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Models2 Feb 2026
- 2026Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies2 Feb 2026
- 2026UEval: A Benchmark for Unified Multimodal Generation29 Jan 2026
- 2026Visual Generation Unlocks Human-Like Reasoning through Multimodal World Models27 Jan 2026
- 2026MMDeepResearch-Bench: A Benchmark for Multimodal Deep Research Agents18 Jan 2026
- 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights1 Dec 2025
- 2025AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Model27 Nov 2025
- 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward25 Nov 2025
- 2026Demystifying Video Reasoningcited by 14
- 2026UEval: A Benchmark for Unified Multimodal Generationcited by 9
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoningcited by 6
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Taskscited by 4
- 2025AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Modelcited by 1
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Modelscited by 1
- 2026DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Modelscited by 1
- 2026The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math?cited by 0
- 2026SketchVLM: Vision language models can annotate images to explain thoughts and guide userscited by 0
- 2026The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Spacecited by 0
- 2026InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Rewardcited by 0
- 2026UniVR: Thinking in Visual Space for Unified Visual Reasoningcited by 0
- 2026VBVR-Pro: A Scalable and Verifiable Suite for Native Visual Reasoning26 Aug 2026
- 2026UniVR: Thinking in Visual Space for Unified Visual Reasoning14 Jul 2026
- 2026DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models9 Jul 2026
- 2026InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward26 May 2026
- 2026The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space11 May 2026
- 2026SketchVLM: Vision language models can annotate images to explain thoughts and guide users23 Apr 2026
- 2026Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks21 Apr 2026
- 2026Pseudo-Unification: Entropy Probing Reveals Divergent Information Patterns in Unified Multimodal Models13 Apr 2026
- 2026Demystifying Video Reasoning17 Mar 2026
- 2026The CompMath-MCQ Dataset: Are LLMs Ready for Higher-Level Math?11 Feb 2026
- 2026UEval: A Benchmark for Unified Multimodal Generation29 Jan 2026
- 2025AIA: Rethinking Architecture Decoupling Strategy In Unified Multimodal Model27 Nov 2025
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 32
- 2026Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomaliescited by 8
- 2026V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learningcited by 0
- 2026V-Rubrics: Visual Faithfulness via Rubric-Based Reinforcement Learning26 Aug 2026
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecture12 May 2026
- 2026Seeing Is Believing? A Benchmark for Multimodal Large Language Models on Visual Illusions and Anomalies2 Feb 2026
- 2026SenseNova-U1.5: Towards Native Unified Visual Intelligence10 Sep 2026
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 12
- 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forwardcited by 10
- 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insightscited by 2
- 2026TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-trainingcited by 2
- 2026Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Modelscited by 2
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?cited by 2
- 2026EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understandingcited by 1
- 2026Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generationcited by 1
- 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generationcited by 0
- 2026What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generationcited by 0
- 2026What Comes Next? Omni-StoryBench for Evaluating Story-Grounded Omnimodal Generation29 Sep 2026
- 2026Thinking in Pictures: A Systematic Benchmark for Reasoning-driven Image Generation2 Sep 2026
- 2026When Does Visual Generation Help Visual Understanding in Unified Multimodal Models?23 Aug 2026
- 2026Open Evaluation Agent: Efficient and Promptable Evaluation of Visual Generative Models10 Aug 2026
- 2026IMUG-Bench: Benchmarking Unified Multimodal Models on Interleaved Understanding and Generation8 Jun 2026
- 2026TorchUMM: A Unified Multimodal Model Codebase for Evaluation, Analysis, and Post-training12 Apr 2026
- 2026EchoGen: Cycle-Consistent Learning for Unified Layout-Image Generation and Understanding18 Mar 2026
- 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Models2 Feb 2026
- 2025Envision: Benchmarking Unified Understanding & Generation for Causal World Process Insights1 Dec 2025
- 2025Does Understanding Inform Generation in Unified Multimodal Models? From Analysis to Path Forward25 Nov 2025
Similar, not cited
Close in wording, no citation either way
- 2026UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?
- 2025ROVER: Benchmarking Reciprocal Cross-Modal Reasoning for Omnimodal Generation
- 2026UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
- 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing
- 2025MM-CoT:A Benchmark for Probing Visual Chain-of-Thought Reasoning in Multimodal Models
- 2025Thinking with Generated Images
- 2026Omni-R1: Towards the Unified Generative Paradigm for Multimodal Reasoning
- 2026Can Unified Generation and Understanding Models Maintain Semantic Equivalence Across Different Output Modalities?