# GaVaMoE: Gaussian-Variational Gated Mixture of Experts for Explainable Recommendation

arXiv: https://arxiv.org/abs/2410.11841 · HTML: https://mlsanity.com/paper/2410.11841/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in MoE for recommender systems, one of 12 directions in Mixture-of-experts.
- Builds on 26 of its 44 references that are in the map.
- First posted on arXiv 15 Oct 2024.

## Placed in

- Mixture-of-experts / [MoE for recommender systems](https://mlsanity.com/moe/moe-for-recommender-systems/)

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding](https://mlsanity.com/paper/2006.16668/) (2020)
- [DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models](https://mlsanity.com/paper/2401.06066/) (2024)
- [ST-MoE: Designing Stable and Transferable Sparse Expert Models](https://mlsanity.com/paper/2202.08906/) (2022)
- [Variational Autoencoders for Collaborative Filtering](https://mlsanity.com/paper/1802.05814/) (2018)
- [Variational Deep Embedding: An Unsupervised and Generative Approach to Clustering](https://mlsanity.com/paper/1611.05148/) (2016)
- [Explainable Recommendation: A Survey and New Perspectives](https://mlsanity.com/paper/1804.11192/) (2018)
- [LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training](https://mlsanity.com/paper/2406.16554/) (2024)
- [Reinforcement Knowledge Graph Reasoning for Explainable Recommendation](https://mlsanity.com/paper/1906.05237/) (2019)
- [Personalized Prompt Learning for Explainable Recommendation](https://mlsanity.com/paper/2202.07371/) (2022)
- [Fairness-Aware Explainable Recommendation over Knowledge Graphs](https://mlsanity.com/paper/2006.02046/) (2020)
- [RecVAE: a New Variational Autoencoder for Top-N Recommendations with Implicit Feedback](https://mlsanity.com/paper/1912.11160/) (2019)
- [Counterfactual Explainable Recommendation](https://mlsanity.com/paper/2108.10539/) (2021)
- [Personalized Transformer for Explainable Recommendation](https://mlsanity.com/paper/2105.11601/) (2021)
- [Neural Rating Regression with Abstractive Tips Generation for Recommendation](https://mlsanity.com/paper/1708.00154/) (2017)
- [Neural Logic Reasoning](https://mlsanity.com/paper/2008.09514/) (2020)
- [Neural Collaborative Reasoning](https://mlsanity.com/paper/2005.08129/) (2020)
- [XRec: Large Language Models for Explainable Recommendation](https://mlsanity.com/paper/2406.02377/) (2024)
- [Multi-Pointer Co-Attention Networks for Recommendation](https://mlsanity.com/paper/1801.09251/) (2018)
- [Factual and Informative Review Generation for Explainable Recommendation](https://mlsanity.com/paper/2209.12613/) (2022)
- [EXTRA: Explanation Ranking Datasets for Explainable Recommendation](https://mlsanity.com/paper/2102.10315/) (2021)

## Built on by (most cited first, 3 of 3)

- [Iterative Semantic Reasoning from Individual to Group Interests for Generative Recommendation with LLMs](https://mlsanity.com/paper/2603.13934/) (2026)
- [DTD-VAE: Disentangled Temporal Dependencies VAE for Credit Risk Prediction](https://mlsanity.com/paper/2608.26473/) (2026)
- [Contrastive Learning for Aspect Representation towards Explainable Recommendation](https://mlsanity.com/paper/2610.07761/) (2026)

## Abstract (arXiv)

Large language model-based explainable recommendation (LLM-based ER) systems show promise in generating human-like explanations for recommendations. However, they face challenges in modeling user-item collaborative preferences, personalizing explanations, and handling sparse user-item interactions. To address these issues, we propose GaVaMoE, a novel Gaussian-Variational Gated Mixture of Experts framework for explainable recommendation. GaVaMoE introduces two key components: (1) a rating reconstruction module that employs Variational Autoencoder (VAE) with a Gaussian Mixture Model (GMM) to capture complex user-item collaborative preferences, serving as a pre-trained multi-gating mechanism; and (2) a set of fine-grained expert models coupled with the multi-gating mechanism for generating highly personalized explanations. The VAE component models latent factors in user-item interactions, while the GMM clusters users with similar behaviors. Each cluster corresponds to a gate in the multi-gating mechanism, routing user-item pairs to appropriate expert models. This architecture enables GaVaMoE to generate tailored explanations for specific user types and preferences, mitigating data sparsity by leveraging user similarities. Extensive experiments on three real-world datasets demonstrate that GaVaMoE significantly outperforms existing methods in explanation quality, personalization, and consistency. Notably, GaVaMoE exhibits robust performance in scenarios with sparse user-item interactions, maintaining high-quality explanations even for users with limited historical data.