# Transformers Meet ACT-R: Repeat-Aware and Sequential Listening Session Recommendation

arXiv: https://arxiv.org/abs/2408.16578 · HTML: https://mlsanity.com/paper/2408.16578/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 89 references that are in the map.
- First posted on arXiv 29 Aug 2024.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Hypergraph Neural Networks](https://mlsanity.com/paper/1809.09401/) (2018)
- [A Survey on Session-based Recommender Systems](https://mlsanity.com/paper/1902.04864/) (2019)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation](https://mlsanity.com/paper/1812.02646/) (2018)
- [ATRank: An Attention-Based User Behavior Modeling Framework for Recommendation](https://mlsanity.com/paper/1711.06632/) (2017)
- [Current Challenges and Visions in Music Recommender Systems Research](https://mlsanity.com/paper/1710.03208/) (2017)
- [Facial Expression Analysis under Partial Occlusion: A Survey](https://mlsanity.com/paper/1802.08784/) (2018)
- [Hierarchical Temporal Convolutional Networks for Dynamic Recommender Systems](https://mlsanity.com/paper/1904.04381/) (2019)
- [Modeling Personalized Item Frequency Information for Next-basket Recommendation](https://mlsanity.com/paper/2006.00556/) (2020)
- [A Semi-Personalized System for User Cold Start Recommendation on Music Streaming Apps](https://mlsanity.com/paper/2106.03819/) (2021)
- [Enhancing Sequential Recommendation with Graph Contrastive Learning](https://mlsanity.com/paper/2205.14837/) (2022)
- [Predicting Temporal Sets with Deep Neural Networks](https://mlsanity.com/paper/2006.11483/) (2020)
- [A Next Basket Recommendation Reality Check](https://mlsanity.com/paper/2109.14233/) (2021)
- [Attention Mixtures for Time-Aware Sequential Recommendation](https://mlsanity.com/paper/2304.08158/) (2023)
- [Improving Collaborative Metric Learning with Efficient Negative Sampling](https://mlsanity.com/paper/1909.10912/) (2019)
- [A Scalable Framework for Automatic Playlist Continuation on Music Streaming Services](https://mlsanity.com/paper/2304.09061/) (2023)
- [Masked and Swapped Sequence Modeling for Next Novel Basket Recommendation in Grocery Shopping](https://mlsanity.com/paper/2308.01308/) (2023)
- [Temporal Effects on Hashtag Reuse in Twitter: A Cognitive-Inspired Hashtag Recommendation Approach](https://mlsanity.com/paper/1701.01276/) (2017)
- [Hierarchical Latent Relation Modeling for Collaborative Metric Learning](https://mlsanity.com/paper/2108.04655/) (2021)
- [Modeling Dynamic Attributes for Next Basket Recommendation](https://mlsanity.com/paper/2109.11654/) (2021)
- [Predicting Music Relistening Behavior Using the ACT-R Framework](https://mlsanity.com/paper/2108.02138/) (2021)
- [Ex2Vec: Characterizing Users and Items from the Mere Exposure Effect](https://mlsanity.com/paper/2311.10635/) (2023)
- [Should we Embed? A Study on the Online Performance of Utilizing Embeddings for Real-Time Job Recommendations](https://mlsanity.com/paper/1907.06556/) (2019)
- [Beyond Accuracy Optimization: On the Value of Item Embeddings for Student Job Recommendations](https://mlsanity.com/paper/1711.07762/) (2017)
- [Modeling Musical Taste Evolution with Recurrent Neural Networks](https://mlsanity.com/paper/1806.06535/) (2018)

## Built on by (most cited first, 4 of 4)

- [Just Ask for Music (JAM): Multimodal and Personalized Natural Language Music Recommendation](https://mlsanity.com/paper/2507.15826/) (2025)
- [Balancing Accuracy and Novelty with Sub-Item Popularity](https://mlsanity.com/paper/2508.05198/) (2025)
- [Modeling Musical Genre Trajectories through Pathlet Learning](https://mlsanity.com/paper/2505.03480/) (2025)
- [Multimodal Music Recommendation System using LLMs](https://mlsanity.com/paper/2606.00125/) (2026)

## Abstract (arXiv)

Music streaming services often leverage sequential recommender systems to predict the best music to showcase to users based on past sequences of listening sessions. Nonetheless, most sequential recommendation methods ignore or insufficiently account for repetitive behaviors. This is a crucial limitation for music recommendation, as repeatedly listening to the same song over time is a common phenomenon that can even change the way users perceive this song. In this paper, we introduce PISA (Psychology-Informed Session embedding using ACT-R), a session-level sequential recommender system that overcomes this limitation. PISA employs a Transformer architecture learning embedding representations of listening sessions and users using attention mechanisms inspired by Anderson's ACT-R (Adaptive Control of Thought-Rational), a cognitive architecture modeling human information access and memory dynamics. This approach enables us to capture dynamic and repetitive patterns from user behaviors, allowing us to effectively predict the songs they will listen to in subsequent sessions, whether they are repeated or new ones. We demonstrate the empirical relevance of PISA using both publicly available listening data from Last.fm and proprietary data from Deezer, a global music streaming service, confirming the critical importance of repetition modeling for sequential listening session recommendation. Along with this paper, we publicly release our proprietary dataset to foster future research in this field, as well as the source code of PISA to facilitate its future use.