mlsanityData to 14 Oct 2026

Structural and Disentangled Adaptation of Large Vision Language Models for Multimodal Recommendation

arXiv 2512.06883 ↗First posted 7 Dec 2025cs.IRcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Multimodal recommendation enhances accuracy by leveraging visual and textual signals, and its success largely depends on learning high-quality cross-modal representations. Recent advances in Large Vision-Language Models (LVLMs) offer unified multimodal representation learning, making them a promising backbone. However, applying LVLMs to recommendation remains challenging due to (i) representation misalignment, where domain gaps between item data and general pre-training lead to unaligned embedding spaces, and (ii) gradient conflicts during fine-tuning, where shared adapters cause interference and a lack of discriminative power. To address this, we propose SDA, a lightweight framework for Structural and Disentangled Adaptation, which integrates two components: Cross-Modal Structural Alignment (CMSA) and Modality-Disentangled Adaptation. CMSA aligns embeddings using intra-modal structures as a soft teacher, while MoDA mitigates gradient conflicts via expertized, gated low-rank paths to disentangle gradient flows. Experiments on three public Amazon datasets show SDA integrates seamlessly with existing multimodal and sequential recommenders, yielding average gains of 6.15% in Hit@10 and 8.64% in NDCG@10. It also achieves up to 12.83% and 18.70% gains on long-tail items with minimal inference overhead. Our code and full experimental results are available at https://github.com/RaoZhongtao/SDA.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 10 of its 15 references that are in the map. First posted on arXiv 7 Dec 2025.

Builds on

10 of 15 references.

References · 10

  1. 2018Self-Attentive Sequential Recommendationcited by 1,184
  2. 2019BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformercited by 964
  3. 2023A Survey on Multimodal Large Language Modelscited by 909
  4. 2015Image-based Recommendations on Styles and Substitutescited by 706
  5. 2015VBPR: Visual Bayesian Personalized Ranking from Implicit Feedbackcited by 151
  6. 2023Multimodal Recommender Systems: A Surveycited by 37
  7. 2023Multi-View Graph Convolutional Network for Multimedia Recommendationcited by 32
  8. 2023MMRec: Simplifying Multimodal Recommendationcited by 17
  9. 2023Attention Calibration for Transformer-based Sequential Recommendationcited by 15
  10. 2022Equivariant Contrastive Learning for Sequential Recommendationcited by 13

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026VLM4Rec: Multimodal Semantic Representation for Recommendation with Large Vision-Language Models
  2. 2025Do Recommender Systems Really Leverage Multimodal Content? A Comprehensive Analysis on Multimodal Representations for Recommendation
  3. 2026Latent-Aligned Reasoning for Multimodal Recommendation
  4. 2024NoteLLM-2: Multimodal Large Representation Models for Recommendation
  5. 2026VLM2Rec: Resolving Modality Collapse in Vision-Language Model Embedders for Multimodal Sequential Recommendation
  6. 2025Multimodal Large Language Models with Adaptive Preference Optimization for Sequential Recommendation
  7. 2025Bridging Domain Gaps between Pretrained Multimodal Models and Recommendations
  8. 2024Rec-GPT4V: Multimodal Recommendation with Large Vision-Language Models