mlsanityData to 7 Oct 2026

STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs

arXiv 2607.17140 ↗First posted 19 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Unified multimodal models (UMMs) aim to integrate visual understanding and generation within a single architecture, but architectural unification alone does not ensure semantic consistency. A model may describe the intended target correctly while generating an inconsistent edit. This exposes an understanding-generation alignment gap: linguistic and visual outputs live in different spaces, yet should be governed by the same target semantics. We study this gap in image editing, where an instruction defines a target state that can be both described and visually realized. Given a source image and an edit instruction, we compare a UMM's target caption with its edited image to test whether the two outputs converge on the same result. Our analysis shows that existing UMMs remain weakly aligned, especially for fine-grained entities, attributes, spatial relations, and local details, indicating that semantic unification is not achieved by architecture alone. To bridge this gap, we propose STBridge, a shared-target alignment framework that connects understanding and generation through a common target state. Here the target caption expresses the desired visual result, while the edited image realizes it visually, replacing separate task-specific paths with a shared information flow from target expression to target realization. STBridge follows an align-then-optimize strategy: supervised fine-tuning first establishes the shared-target channel, and sequential reinforcement learning further refines target-centered coordination. Across visual understanding, image generation, and image editing benchmarks, STBridge consistently improves over the initialization model. Alignment analysis confirms that STBridge narrows the gap between what the model describes and what it generates, demonstrating shared-target alignment as an effective post-training strategy for bridging understanding and generation in UMMs.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 50 of its 52 references that are in the map. First posted on arXiv 19 Jul 2026.

Builds on

50 of 52 references.

References · 50

  1. 2025Qwen3 Technical Reportcited by 8,195
  2. 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
  3. 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
  4. 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
  5. 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
  6. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  7. 2024Emu3: Next-Token Prediction is All You Needcited by 761
  8. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
  9. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  10. 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
  11. 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
  12. 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
38 more
  1. 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
  2. 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
  3. 2023Generative Multimodal Models are In-Context Learnerscited by 446
  4. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  5. 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
  6. 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
  7. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
  8. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  9. 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
  10. 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
  11. 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
  12. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  13. 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
  14. 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
  15. 2023Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenizationcited by 87
  16. 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editingcited by 59
  17. 2025Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editingcited by 58
  18. 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 55
  19. 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
  20. 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 48
  21. 2023VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generationcited by 41
  22. 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
  23. 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
  24. 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
  25. 2025UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learningcited by 23
  26. 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 23
  27. 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
  28. 2026Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generationcited by 19
  29. 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 17
  30. 2025Unified Multimodal Models as Auto-Encoderscited by 13
  31. 2026ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editingcited by 10
  32. 2026Leveraging Verifier-Based Reinforcement Learning in Image Editingcited by 8
  33. 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 6
  34. 2026Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Modelscited by 5
  35. 2026OddGridBench: Exposing the Lack of Fine-Grained Visual Discrepancy Sensitivity in Multimodal Large Language Modelscited by 4
  36. 2026Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimizationcited by 4
  37. 2026Steering Visual Generation in Unified Multimodal Models with Understanding Supervisioncited by 3
  38. 2026LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generationcited by 1

Built on by

Not cited yet.