STBridge: Shared-Target Alignment for Bridging Understanding and Generation in UMMs
Not on the map. Why?
Abstract (arXiv)
Unified multimodal models (UMMs) aim to integrate visual understanding and generation within a single architecture, but architectural unification alone does not ensure semantic consistency. A model may describe the intended target correctly while generating an inconsistent edit. This exposes an understanding-generation alignment gap: linguistic and visual outputs live in different spaces, yet should be governed by the same target semantics. We study this gap in image editing, where an instruction defines a target state that can be both described and visually realized. Given a source image and an edit instruction, we compare a UMM's target caption with its edited image to test whether the two outputs converge on the same result. Our analysis shows that existing UMMs remain weakly aligned, especially for fine-grained entities, attributes, spatial relations, and local details, indicating that semantic unification is not achieved by architecture alone. To bridge this gap, we propose STBridge, a shared-target alignment framework that connects understanding and generation through a common target state. Here the target caption expresses the desired visual result, while the edited image realizes it visually, replacing separate task-specific paths with a shared information flow from target expression to target realization. STBridge follows an align-then-optimize strategy: supervised fine-tuning first establishes the shared-target channel, and sequential reinforcement learning further refines target-centered coordination. Across visual understanding, image generation, and image editing benchmarks, STBridge consistently improves over the initialization model. Alignment analysis confirms that STBridge narrows the gap between what the model describes and what it generates, demonstrating shared-target alignment as an effective post-training strategy for bridging understanding and generation in UMMs.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 50 of its 52 references that are in the map. First posted on arXiv 19 Jul 2026.
Builds on
50 of 52 references.
References · 50
- 2025Qwen3 Technical Reportcited by 8,195
- 2023MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilitiescited by 1,221
- 2024Chameleon: Mixed-Modal Early-Fusion Foundation Modelscited by 904
- 2025Emerging Properties in Unified Multimodal Pretrainingcited by 855
- 2024Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMscited by 822
- 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
- 2024Emu3: Next-Token Prediction is All You Needcited by 761
- 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 742
- 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
- 2024Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMscited by 719
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2024BLINK: Multimodal Large Language Models Can See but Not Perceivecited by 529
38 more
- 2024Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Modelcited by 468
- 2024Janus: Decoupling Visual Encoding for Unified Multimodal Understanding and Generationcited by 447
- 2023Generative Multimodal Models are In-Context Learnerscited by 446
- 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
- 2024SEED-X: Multimodal Models with Unified Multi-granularity Comprehension and Generationcited by 328
- 2023Unified-IO 2: Scaling Autoregressive Multimodal Models with Vision, Language, Audio, and Actioncited by 301
- 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 296
- 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
- 2025Show-o2: Improved Native Unified Multimodal Modelscited by 242
- 2025ImgEdit: A Unified Image Editing Dataset and Benchmarkcited by 225
- 2024MetaMorph: Multimodal Understanding and Generation via Instruction Tuningcited by 205
- 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
- 2025MixGRPO: Unlocking Flow-based GRPO Efficiency with Mixed ODE-SDEcited by 168
- 2025Envisioning Beyond the Pixels: Benchmarking Reasoning-Informed Visual Editingcited by 97
- 2023Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenizationcited by 87
- 2025EditReward: A Human-Aligned Reward Model for Instruction-Guided Image Editingcited by 59
- 2025Pico-Banana-400K: A Large-Scale Dataset for Text-Guided Image Editingcited by 58
- 2025ThinkMorph: Emergent Properties in Multimodal Interleaved Chain-of-Thought Reasoningcited by 55
- 2025OneCAT: Decoder-Only Auto-Regressive Model for Unified Understanding and Generationcited by 50
- 2025Reconstruction Alignment Improves Unified Multimodal Modelscited by 48
- 2023VL-GPT: A Generative Pre-trained Transformer for Vision and Language Understanding and Generationcited by 41
- 2025TUNA: Taming Unified Visual Representations for Native Unified Multimodal Modelscited by 34
- 2025UniGen: Enhanced Training & Test-Time Strategies for Unified Multimodal Understanding and Generationcited by 32
- 2026SenseNova-U1: Unifying Multimodal Understanding and Generation with NEO-unify Architecturecited by 27
- 2025UniRL: Self-Improving Unified Multimodal Models via Supervised and Reinforcement Learningcited by 23
- 2025EditThinker: Unlocking Iterative Reasoning for Any Image Editorcited by 23
- 2026InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editingcited by 23
- 2026Generation Enhances Understanding in Unified Multimodal Models via Multi-Representation Generationcited by 19
- 2026Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding and Generationcited by 17
- 2025Unified Multimodal Models as Auto-Encoderscited by 13
- 2026ThinkRL-Edit: Thinking in Reinforcement Learning for Reasoning-Centric Image Editingcited by 10
- 2026Leveraging Verifier-Based Reinforcement Learning in Image Editingcited by 8
- 2026LaViDa-R1: Advancing Reasoning for Unified Multimodal Diffusion Language Modelscited by 6
- 2026Learning to Generate via Understanding: Understanding-Driven Intrinsic Rewarding for Unified Multimodal Modelscited by 5
- 2026OddGridBench: Exposing the Lack of Fine-Grained Visual Discrepancy Sensitivity in Multimodal Large Language Modelscited by 4
- 2026Towards Unified Multimodal Interleaved Generation via Group Relative Policy Optimizationcited by 4
- 2026Steering Visual Generation in Unified Multimodal Models with Understanding Supervisioncited by 3
- 2026LVRPO: Language-Visual Alignment with GRPO for Multimodal Understanding and Generationcited by 1
Built on by
Not cited yet.