mlsanityData to 14 Oct 2026

Steering Visual Generation in Unified Multimodal Models with Understanding Supervision

arXiv 2605.05781 ↗First posted 7 May 2026cs.CV cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Unified multimodal models are envisioned to bridge the gap between understanding and generation. Yet, to achieve competitive performance, state-of-the-art models adopt largely decoupled understanding and generation components. This design, while effective for individual tasks, weakens the connection required for mutual enhancement, leaving the potential synergy empirically uncertain. We propose to explicitly restore this synergy by introducing Understanding-Oriented Post-Training (UNO), a lightweight framework that treats understanding not only as a distinct task, but also a direct supervisory signal to steer generative representations. By incorporating objectives that encode semantic abstraction (captioning) and structural details (visual regression), we enable effective gradient flow from understanding to generation. Extensive experiments on image generation and editing demonstrate that understanding can serve as an effective catalyst for generation.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 66 of its 68 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

66 of 68 references.

References · 66

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2023Visual Instruction Tuningcited by 7,869
  6. 2022Scalable Diffusion Models with Transformerscited by 6,054
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  9. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  10. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  12. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
38 more
  1. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,652
  2. 2025SigLIP 2: Multilingual Vision-Language Encoders with Improved Semantic Understanding, Localization, and Dense Featurescited by 1,188
  3. 2025FLUX.1 Kontext: Flow Matching for In-Context Image Generation and Editing in Latent Spacecited by 1,006
  4. 2023GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignmentcited by 972
  5. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 927
  6. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 897
  7. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 812
  8. 2024Emu3: Next-Token Prediction is All You Needcited by 781
  9. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 752
  10. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 703
  11. 2024Autoregressive Image Generation without Vector Quantizationcited by 651
  12. 2023MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editingcited by 624
  13. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignmentcited by 475
  14. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 457
  15. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  16. 2025Step1X-Edit: A Practical Framework for General Image Editingcited by 429
  17. 2024OmniGen: Unified Image Generationcited by 394
  18. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 376
  19. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 331
  20. 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 304
  21. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  22. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 295
  23. 2025Diffusion Transformers with Representation Autoencoderscited by 295
  24. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 250
  25. 2025Transfer between Modalities with MetaQueriescited by 236
  26. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 231
  27. 2025REPA-E: Unlocking VAE for End-to-End Tuning with Latent Diffusion Transformerscited by 225
  28. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 206
  29. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  30. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 205
  31. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 204
  32. 2024Mixture-of-Transformers: A Sparse and Scalable Architecture for Multi-Modal Foundation Modelscited by 163
  33. 2024JanusFlow: Harmonizing Autoregression and Rectified Flow for Unified Multimodal Understanding and Generationcited by 163
  34. 2025Mogao: An Omni Foundation Model for Interleaved Multi-Modal Generationcited by 109
  35. 2025EQ-VAE: Equivariance Regularized Latent Space for Improved Generative Image Modelingcited by 89
  36. 2025Latent Diffusion Model without Variational Autoencodercited by 88
  37. 2025AnyEdit: Edit Any Knowledge Encoded in Language Modelscited by 76
  38. 2025Thinking with Generated Imagescited by 73

50 most cited of 66 All 66 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.