DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement
Not on the map. Why?
Abstract (arXiv)
Unified Multimodal models (UMMs) built on a single architecture have shown impressive performance in both understanding and generation. We identify a fundamental challenge that lies in inductive biases induced by distinct supervision signals: generation branch prefers high-fidelity, fine-grained representations capable of reconstruction, while the understanding favours semantically discriminative embeddings that remain invariant to task-irrelevant factors. Consequently, optimizing these complementary but non-equivalent objectives within a monolithic backbone leads to mutual impairment instead of enhancement. In this paper, we first analyze the root cause of this interference in unified backbones and reveal a complementary structure in their internal representations. Motivated by the observation, we propose DIVA, a self-improved post-training framework that transforms the representation divergence into interior synergy. By explicitly factorizing the visual representation into shared and unique components based on two complementary information flow, DIVA enables both the understanding and generation branches to achieve beneficial transferring while preserving the integrity of unique information from cross-flow interference via mutual information estimation. Despite its generality, our method consistently achieves improvements across visual understanding (+7.82%) and generation (+8.46%). The official code is available at: https://github.com/Jayyy-H/DIVA.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 46 of its 51 references that are in the map. First posted on arXiv 25 May 2026.
Builds on
46 of 51 references.
References · 46
- 2023Visual Instruction Tuningcited by 7,792
- 2025Qwen2.5-VL Technical Reportcited by 5,418
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
- 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
- 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
- 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
34 more
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2025Qwen-Image Technical Reportcited by 973
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 960
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 946
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 455
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 417
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
- 2024UltraEdit: Instruction-based Fine-Grained Image Editing at Scalecited by 266
- 2024AnyEdit: Mastering Unified High-Quality Image Editing for Any Ideacited by 235
- 2020CLUB: A Contrastive Log-ratio Upper Bound of Mutual Informationcited by 233
- 2025Transfer between Modalities with MetaQueriescited by 231
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
- 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
- 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
- 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 108
- 2023CapsFusion: Rethinking Image-Text Data at Scalecited by 90
- 2023Factorized Contrastive Learning: Going Beyond Multi-view Redundancycited by 73
- 2025Breaking the Modality Barrier: Universal Embedding Learning with Multimodal LLMscited by 73
- 2025GPT-IMAGE-EDIT-1.5M: A Million-Scale, GPT-Generated Image Datasetcited by 68
- 2024Liquid: Language Models are Scalable and Unified Multi-modal Generatorscited by 62
- 2025NextStep-1: Toward Autoregressive Image Generation with Continuous Tokens at Scalecited by 55
- 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 48
- 2024MonoFormer: One Transformer for Both Diffusion and Autoregressioncited by 45
- 2025Generative Multimodal Pretraining with Discrete Diffusion Timestep Tokenscited by 27
Built on by
Not cited yet.