# Structural and Disentangled Adaptation of Large Vision Language Models for Multimodal Recommendation

arXiv: https://arxiv.org/abs/2512.06883 · HTML: https://mlsanity.com/paper/2512.06883/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 7 Dec 2025.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 10 of 10)

- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [VBPR: Visual Bayesian Personalized Ranking from Implicit Feedback](https://mlsanity.com/paper/1510.01784/) (2015)
- [Multimodal Recommender Systems: A Survey](https://mlsanity.com/paper/2302.03883/) (2023)
- [Multi-View Graph Convolutional Network for Multimedia Recommendation](https://mlsanity.com/paper/2308.03588/) (2023)
- [MMRec: Simplifying Multimodal Recommendation](https://mlsanity.com/paper/2302.03497/) (2023)
- [Attention Calibration for Transformer-based Sequential Recommendation](https://mlsanity.com/paper/2308.09419/) (2023)
- [Equivariant Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2211.05290/) (2022)

## Abstract (arXiv)

Multimodal recommendation enhances accuracy by leveraging visual and textual signals, and its success largely depends on learning high-quality cross-modal representations. Recent advances in Large Vision-Language Models (LVLMs) offer unified multimodal representation learning, making them a promising backbone. However, applying LVLMs to recommendation remains challenging due to (i) representation misalignment, where domain gaps between item data and general pre-training lead to unaligned embedding spaces, and (ii) gradient conflicts during fine-tuning, where shared adapters cause interference and a lack of discriminative power. To address this, we propose SDA, a lightweight framework for Structural and Disentangled Adaptation, which integrates two components: Cross-Modal Structural Alignment (CMSA) and Modality-Disentangled Adaptation. CMSA aligns embeddings using intra-modal structures as a soft teacher, while MoDA mitigates gradient conflicts via expertized, gated low-rank paths to disentangle gradient flows. Experiments on three public Amazon datasets show SDA integrates seamlessly with existing multimodal and sequential recommenders, yielding average gains of 6.15% in Hit@10 and 8.64% in NDCG@10. It also achieves up to 12.83% and 18.70% gains on long-tail items with minimal inference overhead. Our code and full experimental results are available at https://github.com/RaoZhongtao/SDA.