# Revisiting scalable sequential recommendation with Multi-Embedding Approach and Mixture-of-Experts

arXiv: https://arxiv.org/abs/2510.25285 · HTML: https://mlsanity.com/paper/2510.25285/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in MoE for recommender systems, one of 12 directions in Mixture-of-experts.
- Builds on 33 of its 67 references that are in the map.
- First posted on arXiv 29 Oct 2025.

## Placed in

- Mixture-of-experts / [MoE for recommender systems](https://mlsanity.com/moe/moe-for-recommender-systems/)

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models](https://mlsanity.com/paper/2401.06066/) (2024)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [A Survey on Large Language Models for Recommendation](https://mlsanity.com/paper/2305.19860/) (2023)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [OneRec: Unifying Retrieve and Rank with Generative Recommender and Iterative Preference Alignment](https://mlsanity.com/paper/2502.18965/) (2025)
- [GME: Improving Universal Multimodal Retrieval by Multimodal LLMs](https://mlsanity.com/paper/2412.16855/) (2024)
- [Recurrent Neural Networks with Top-k Gains for Session-based Recommendations](https://mlsanity.com/paper/1706.03847/) (2017)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Intent Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2202.02519/) (2022)
- [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://mlsanity.com/paper/2403.02545/) (2024)
- [Entropy Law: The Story Behind Data Compression and LLM Performance](https://mlsanity.com/paper/2407.06645/) (2024)
- [MCNE: An End-to-End Framework for Learning Multiple Conditional Network Representations of Social Network](https://mlsanity.com/paper/1905.11013/) (2019)
- [Turning Dross Into Gold Loss: is BERT4Rec really better than SASRec?](https://mlsanity.com/paper/2309.07602/) (2023)
- [RAPID: Efficient Retrieval-Augmented Long Text Generation with Writing Planning and Information Discovery](https://mlsanity.com/paper/2503.00751/) (2025)
- [OneSug: The Unified End-to-End Generative Framework for E-commerce Query Suggestion](https://mlsanity.com/paper/2506.06913/) (2025)
- [Scaling New Frontiers: Insights into Large Recommendation Models](https://mlsanity.com/paper/2412.00714/) (2024)
- [TD3: Tucker Decomposition Based Dataset Distillation Method for Sequential Recommendation](https://mlsanity.com/paper/2502.02854/) (2025)
- [Generative Large Recommendation Models: Emerging Trends in LLMs for Recommendation](https://mlsanity.com/paper/2502.13783/) (2025)
- [Breaking Determinism: Fuzzy Modeling of Sequential Recommendation Using Discrete State Space Diffusion Model](https://mlsanity.com/paper/2410.23994/) (2024)
- [APGL4SR: A Generic Framework with Adaptive and Personalized Global Collaborative Information in Sequential Recommendation](https://mlsanity.com/paper/2311.02816/) (2023)
- [TiM4Rec: An Efficient Sequential Recommendation Model Based on Time-Aware Structured State Space Duality Model](https://mlsanity.com/paper/2409.16182/) (2024)
- [Optimizing Sequential Recommendation Models with Scaling Laws and Approximate Entropy](https://mlsanity.com/paper/2412.00430/) (2024)
- [GUESR: A Global Unsupervised Data-Enhancement with Bucket-Cluster Sampling for Sequential Recommendation](https://mlsanity.com/paper/2303.00243/) (2023)
- [Multi-granularity Interest Retrieval and Refinement Network for Long-Term User Behavior Modeling in CTR Prediction](https://mlsanity.com/paper/2411.15005/) (2024)

## Built on by (most cited first, 3 of 3)

- [Why Thinking Hurts: Diagnosing and Rectifying Linguistic Inertia in Large Language Models for Recommendation](https://mlsanity.com/paper/2602.16587/) (2026)
- [Efficient Personalized Reranking with Semi-Autoregressive Generation and Online Knowledge Distillation](https://mlsanity.com/paper/2603.07107/) (2026)
- [AsymSpec: Context-Asymmetric Speculative Decoding for Agentic LLMs](https://mlsanity.com/paper/2608.26004/) (2026)

## Abstract (arXiv)

In recommendation systems, how to effectively scale up recommendation models has been an essential research topic. While significant progress has been made in developing advanced and scalable architectures for sequential recommendation(SR) models, there are still challenges due to items' multi-faceted characteristics and dynamic item relevance in the user context. To address these issues, we propose Fuxi-MME, a framework that integrates a multi-embedding strategy with a Mixture-of-Experts (MoE) architecture. Specifically, to efficiently capture diverse item characteristics in a decoupled manner, we decompose the conventional single embedding matrix into several lower-dimensional embedding matrices. Additionally, by substituting relevant parameters in the Fuxi Block with an MoE layer, our model achieves adaptive and specialized transformation of the enriched representations. Empirical results on public datasets show that our proposed framework outperforms several competitive baselines.