Steering Visual Generation in Unified Multimodal Models with Understanding Supervision
Not on the map. Why?
Abstract (arXiv)
Unified multimodal models are envisioned to bridge the gap between understanding and generation. Yet, to achieve competitive performance, state-of-the-art models adopt largely decoupled understanding and generation components. This design, while effective for individual tasks, weakens the connection required for mutual enhancement, leaving the potential synergy empirically uncertain. We propose to explicitly restore this synergy by introducing Understanding-Oriented Post-Training (UNO), a lightweight framework that treats understanding not only as a distinct task, but also a direct supervisory signal to steer generative representations. By incorporating objectives that encode semantic abstraction (captioning) and structural details (visual regression), we enable effective gradient flow from understanding to generation. Extensive experiments on image generation and editing demonstrate that understanding can serve as an effective catalyst for generation.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 66 of its 68 references that are in the map. First posted on arXiv 7 May 2026.
Builds on
66 of 68 references.
References · 66
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2023GPT-4 Technical Reportcited by 18,396
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2025Qwen3 Technical Reportcited by 8,820
- 2023Visual Instruction Tuningcited by 7,869
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2025Qwen2.5-VL Technical Reportcited by 5,587
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
38 more
- 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
- 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
- 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
- 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
- 2024Emu3: Next-Token Prediction is All You Needcited by 781
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
- 2024Autoregressive Image Generation without Vector Quantizationcited by 651
- 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
- 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
- 2024OmniGen: Unified Image Generationcited by 394
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
- 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
- 2025Diffusion Transformers with Representation Autoencoderscited by 295
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
- 2025Transfer between Modalities with MetaQueriescited by 236
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
- 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
- 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
- 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
- 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
- 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 163
- 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
- 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 89
- 2025Latent Diffusion Model without Variational Autoencodercited by 88
- 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 76
- 2025Thinking with Generated Imagescited by 73
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2026: 3.
3 papers.
- 2026Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Modelscited by 3
- 2026Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to Systemcited by 2
- 2026STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMscited by 0
- 2026Dreaming in Flow: Generative Grounding Feedback for Self-Evolving Unified Multimodal Models8 Sep 2026
- 2026Uncovering Understanding-Generation Synergy in Native Unified Multimodal Models: From Representation, Task to System1 Sep 2026
- 2026STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs19 Jul 2026