mlsanityData to 7 Oct 2026

Do Unified Multimodal Models Think in One Space? A Lens Through Cross-Branch Steering

arXiv 2607.26411 ↗First posted 29 Jul 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Unified multimodal models (UMMs) aim to integrate understanding and generation within a single architecture, yet it remains unclear whether these capabilities share a unified and transferable semantic space. This question is fundamentally challenging, as the two branches operate over heterogeneous representations (text tokens vs. visual latents) and distinct training objectives, making direct comparison difficult. To address this, we introduce cross-branch semantic steering, an intervention-based framework that extracts semantic directions from one branch and applies them to the other. We show that steering vectors learned from the understanding branch can transfer to generation, enabling controllable image synthesis and improved semantic faithfulness. In contrast, the reverse direction consistently shows limited effectiveness. Our analysis suggests that this asymmetry may be related to a practical representational mismatch: understanding-derived vectors capture transferable, object-centric semantics, while generation-derived vectors primarily encode low-level appearance features. Our results reveal that architectural unification does not guarantee semantic alignment, and establish cross-branch steering as a practical tool for probing multimodal representations.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 38 references that are in the map. First posted on arXiv 29 Jul 2026.

Builds on

27 of 38 references.

References · 27

  1. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  2. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  3. 2025Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scalingcited by 787
  4. 2025BLIP3-o: A Family of Fully Open Unified Multimodal Models-Architecture, Training and Datasetcited by 363
  5. 2024TokenFlow: Unified Image Tokenizer for Multimodal Understanding and Generationcited by 200
  6. 2025WISE: A World Knowledge-Informed Semantic Evaluation for Text-to-Image Generationcited by 196
  7. 2024Analyzing the Generalization and Reliability of Steering Vectorscited by 130
  8. 2024ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficultycited by 60
  9. 2025UniToken: Harmonizing Multimodal Understanding and Generation through Unified Visual Encodingcited by 49
  10. 2024Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreakscited by 44
  11. 2024Steering Rectified Flow Models in the Vector Field for Controlled Image Generationcited by 38
  12. 2025Evaluating and Steering Modality Preferences in Multimodal Large Language Modelcited by 33
15 more
  1. 2025Sce2DriveX: A Generalized MLLM Framework for Scene-to-Drive Learningcited by 25
  2. 2025Automating Steering for Safe Multimodal Large Language Modelscited by 17
  3. 2025Learning to Steer: Input-dependent Steering for Multimodal LLMscited by 13
  4. 2025EasySteer: A Unified Framework for High-Performance and Extensible LLM Steeringcited by 13
  5. 2026Controllable LLM Reasoning via Sparse Autoencoder-Based Steeringcited by 12
  6. 2025UniFork: Exploring Modality Alignment for Unified Multimodal Understanding and Generationcited by 11
  7. 2026Quantifying the Gap between Understanding and Generation within Unified Multimodal Modelscited by 11
  8. 2025Enhancing LLM Steering through Sparse Autoencoder-Based Vector Refinementcited by 9
  9. 2025UniGame: Turning a Unified Multimodal Model Into Its Own Adversarycited by 9
  10. 2025Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Modelscited by 8
  11. 2025Steering Guidance for Personalized Text-to-Image Diffusion Modelscited by 7
  12. 2025Steering Multimodal Large Language Models Decoding for Context-Aware Safetycited by 6
  13. 2026Steering Large Reasoning Models towards Concise Reasoning via Flow Matchingcited by 3
  14. 2026Rethinking UMM Visual Generation: Masked Modeling for Efficient Image-Only Pre-trainingcited by 1
  15. 2026Understanding the Theoretical Foundations of Deep Neural Networks through Differential Equationscited by 1

Built on by

Not cited yet.