# Train Once, Deploy Anywhere: Matryoshka Representation Learning for Multimodal Recommendation

arXiv: https://arxiv.org/abs/2409.16627 · HTML: https://mlsanity.com/paper/2409.16627/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Sep 2024.

## Built on by (most cited first, 9 of 9)

- [Towards Generalizable Robotic Manipulation in Dynamic Environments](https://mlsanity.com/paper/2603.15620/) (2026)
- [Unlocking Multimodal Document Intelligence: From Current Triumphs to Future Frontiers of Visual Document Retrieval](https://mlsanity.com/paper/2602.19961/) (2026)
- [Training Matryoshka Mixture-of-Experts for Elastic Inference-Time Expert Utilization](https://mlsanity.com/paper/2509.26520/) (2025)
- [Retrieval Augmented Conversational Recommendation with Reinforcement Learning](https://mlsanity.com/paper/2604.04457/) (2026)
- [Matryoshka Model Learning for Improved Elastic Student Models](https://mlsanity.com/paper/2505.23337/) (2025)
- [MM-Matryoshka: Towards Budget-Elastic Visual Document Retrieval via a 2D Multimodal Matryoshka Training Framework](https://mlsanity.com/paper/2606.07654/) (2026)
- [Hyperspectral Image Fusion with Spectral-Band and Fusion-Scale Agnosticism](https://mlsanity.com/paper/2602.01681/) (2026)
- [Transferable Sequential Recommendation via Vector Quantized Meta Learning](https://mlsanity.com/paper/2411.01785/) (2024)
- [Eigenfunction Extraction for Ordered Representation Learning](https://mlsanity.com/paper/2510.24672/) (2025)

## Abstract (arXiv)

Despite recent advancements in language and vision modeling, integrating rich multimodal knowledge into recommender systems continues to pose significant challenges. This is primarily due to the need for efficient recommendation, which requires adaptive and interactive responses. In this study, we focus on sequential recommendation and introduce a lightweight framework called full-scale Matryoshka representation learning for multimodal recommendation (fMRLRec). Our fMRLRec captures item features at different granularities, learning informative representations for efficient recommendation across multiple dimensions. To integrate item features from diverse modalities, fMRLRec employs a simple mapping to project multimodal item features into an aligned feature space. Additionally, we design an efficient linear transformation that embeds smaller features into larger ones, substantially reducing memory requirements for large-scale training on recommendation data. Combined with improved state space modeling techniques, fMRLRec scales to different dimensions and only requires one-time training to produce multiple models tailored to various granularities. We demonstrate the effectiveness and efficiency of fMRLRec on multiple benchmark datasets, which consistently achieves superior performance over state-of-the-art baseline methods. We make our code and data publicly available at https://github.com/yueqirex/fMRLRec.