# Attention Mixtures for Time-Aware Sequential Recommendation

arXiv: https://arxiv.org/abs/2304.08158 · HTML: https://mlsanity.com/paper/2304.08158/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 60 references that are in the map.
- First posted on arXiv 17 Apr 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned](https://mlsanity.com/paper/1905.09418/) (2019)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1811.00855/) (2018)
- [A Survey on Session-based Recommender Systems](https://mlsanity.com/paper/1902.04864/) (2019)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [Self-attention with Functional Time Representation Learning](https://mlsanity.com/paper/1911.12864/) (2019)
- [RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation](https://mlsanity.com/paper/1812.02646/) (2018)
- [ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation](https://mlsanity.com/paper/1711.06632/) (2017)
- [Rethinking the Item Order in Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1911.11942/) (2019)
- [Current Challenges and Visions in Music Recommender Systems Research](https://mlsanity.com/paper/1710.03208/) (2017)
- [Multi-Behavior Hypergraph-Enhanced Transformer for Sequential Recommendation](https://mlsanity.com/paper/2207.05584/) (2022)
- [Latent Relational Metric Learning via Memory-based Attention for Collaborative Ranking](https://mlsanity.com/paper/1707.05176/) (2017)
- [Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems](https://mlsanity.com/paper/1904.04381/) (2019)
- [Modeling Personalized Item Frequency Information for Next-basket Recommendation](https://mlsanity.com/paper/2006.00556/) (2020)
- [CARCA: Context and Attribute-Aware Next-Item Recommendation via Cross-Attention](https://mlsanity.com/paper/2204.06519/) (2022)
- [Multi-Behavior Sequential Recommendation with Temporal Graph Transformer](https://mlsanity.com/paper/2206.02687/) (2022)
- [Enhancing Sequential Recommendation with Graph Contrastive Learning](https://mlsanity.com/paper/2205.14837/) (2022)
- [MEANTIME: Mixture of Attention Mechanisms with Multi-temporal Embeddings for Sequential Recommendation](https://mlsanity.com/paper/2008.08273/) (2020)
- [Collaborative Translational Metric Learning](https://mlsanity.com/paper/1906.01637/) (2019)
- [Effective and Efficient Training for Sequential Recommendation using Recency Sampling](https://mlsanity.com/paper/2207.02643/) (2022)
- [Flow Moods: Recommending Music by Moods on Deezer](https://mlsanity.com/paper/2207.11229/) (2022)
- [Contrastive Learning with Bidirectional Transformers for Sequential Recommendation](https://mlsanity.com/paper/2208.03895/) (2022)
- [FairSR: Fairness-aware Sequential Recommendation through Multi-Task Learning with Preference Graph Embeddings](https://mlsanity.com/paper/2205.00313/) (2022)
- [Learning Self-Modulating Attention in Continuous Time Space with Applications to Sequential Recommendation](https://mlsanity.com/paper/2204.06517/) (2022)
- [Learning Heterogeneous Temporal Patterns of User Preference for Timely Recommendation](https://mlsanity.com/paper/2104.14200/) (2021)
- [Eigen Analysis of Self-Attention and its Reconstruction from Partial Computation](https://mlsanity.com/paper/2106.08823/) (2021)

## Built on by (most cited first, 13 of 13)

- [Sparse Meets Dense: Unified Generative Recommendations with Cascaded Sparse-Dense Representations](https://mlsanity.com/paper/2503.02453/) (2025)
- [Hierarchical Time-Aware Mixture of Experts for Multi-Modal Sequential Recommendation](https://mlsanity.com/paper/2501.14269/) (2025)
- [Retrieval of Temporal Event Sequences from Textual Descriptions](https://mlsanity.com/paper/2410.14043/) (2024)
- [UniRec: A Dual Enhancement of Uniformity and Frequency in Sequential Recommendations](https://mlsanity.com/paper/2406.18470/) (2024)
- [Transformers Meet ACT-R: Repeat-Aware and Sequential Listening Session Recommendation](https://mlsanity.com/paper/2408.16578/) (2024)
- [Balancing Accuracy and Novelty with Sub-Item Popularity](https://mlsanity.com/paper/2508.05198/) (2025)
- [When and What to Recommend: Joint Modeling of Timing and Content for Active Sequential Recommendation](https://mlsanity.com/paper/2511.18717/) (2025)
- [T-RoPE: Time-Aware Rotary Position Embedding for Sequential Recommendation](https://mlsanity.com/paper/2609.30576/) (2026)
- [Sequential Recommendation on Temporal Proximities with Contrastive Learning and Self-Attention](https://mlsanity.com/paper/2402.09784/) (2024)
- [TME-PSR: Time-aware, Multi-interest, and Explanation Personalization for Sequential Recommendation](https://mlsanity.com/paper/2604.09439/) (2026)
- [Learning Behaviorally Grounded Item Embeddings via Personalized Temporal Contexts](https://mlsanity.com/paper/2604.15581/) (2026)
- [Uniboost: Global Coordination with Value Alignment for Fair and Efficient Traffic Allocation](https://mlsanity.com/paper/2605.26424/) (2026)
- [The Pitfall of Scaling Up: Uncovering and Mitigating Popularity Bias Amplification in Scaling Transformer-based Recommenders](https://mlsanity.com/paper/2606.21911/) (2026)

## Abstract (arXiv)

Transformers emerged as powerful methods for sequential recommendation. However, existing architectures often overlook the complex dependencies between user preferences and the temporal context. In this short paper, we introduce MOJITO, an improved Transformer sequential recommender system that addresses this limitation. MOJITO leverages Gaussian mixtures of attention-based temporal context and item embedding representations for sequential modeling. Such an approach permits to accurately predict which items should be recommended next to users depending on past actions and the temporal context. We demonstrate the relevance of our approach, by empirically outperforming existing Transformers for sequential recommendation on several real-world datasets.