# DenseRec: Revisiting Dense Content Embeddings for Sequential Transformer-based Recommendation

arXiv: https://arxiv.org/abs/2508.18442 · HTML: https://mlsanity.com/paper/2508.18442/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 37 references that are in the map.
- First posted on arXiv 25 Aug 2025.

## Builds on (most cited first, 19 of 19)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Text Is All You Need: Learning Language Representations for Sequential Recommendation](https://mlsanity.com/paper/2305.13731/) (2023)
- [How Can Recommender Systems Benefit from Large Language Models: A Survey](https://mlsanity.com/paper/2306.05817/) (2023)
- [Improved Recurrent Neural Networks for Session-based Recommendations](https://mlsanity.com/paper/1606.08117/) (2016)
- [Better Generalization with Semantic IDs: A Case Study in Ranking for Recommendations](https://mlsanity.com/paper/2306.08121/) (2023)
- [gSASRec: Reducing Overconfidence in Sequential Recommendation Trained with Negative Sampling](https://mlsanity.com/paper/2308.07192/) (2023)
- [Large Language Models as Recommender Systems: A Study of Popularity Bias](https://mlsanity.com/paper/2406.01285/) (2024)
- [Ask the GRU: Multi-Task Learning for Deep Text Recommendations](https://mlsanity.com/paper/1609.02116/) (2016)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Unifying Generative and Dense Retrieval for Sequential Recommendation](https://mlsanity.com/paper/2411.18814/) (2024)
- [Scaling Recommender Transformers to One Billion Parameters](https://mlsanity.com/paper/2507.15994/) (2025)
- [PinFM: Foundation Model for User Activity Sequences at a Billion-scale Visual Discovery Platform](https://mlsanity.com/paper/2507.12704/) (2025)
- [Enhancing Sequential Music Recommendation with Personalized Popularity Awareness](https://mlsanity.com/paper/2409.04329/) (2024)
- [Building a Scalable, Effective, and Steerable Search and Ranking Platform](https://mlsanity.com/paper/2409.02856/) (2024)
- [Ranking Across Different Content Types: The Robust Beauty of Multinomial Blending](https://mlsanity.com/paper/2408.09168/) (2024)

## Built on by (most cited first, 2 of 2)

- [Cross-Representation Knowledge Transfer for Improved Sequential Recommendations](https://mlsanity.com/paper/2602.23471/) (2026)
- [Balancing Trial and Reorder: A Hybrid Sequential Transformer-GBDT Ranker for On-Demand Delivery](https://mlsanity.com/paper/2609.16407/) (2026)

## Abstract (arXiv)

Transformer-based sequential recommenders, such as SASRec or BERT4Rec, typically rely solely on learned item ID embeddings, making them vulnerable to the item cold-start problem, particularly in environments with dynamic item catalogs. While dense content embeddings from pre-trained models offer potential solutions, direct integration into transformer-based recommenders has consistently underperformed compared to ID-only approaches. We revisit this integration challenge and propose DenseRec, a simple yet effective method that introduces a dual-path embedding approach. DenseRec learns a linear projection from the dense embedding space into the ID embedding space during training, enabling seamless generalization to previously unseen items without requiring specialized embedding models or complex infrastructure. In experiments on three real-world datasets, we find DenseRec to consistently outperform an ID-only SASRec baseline, even without additional hyperparameter tuning and while using compact embedding models. Our analysis suggests improvements primarily arise from better sequence representations in the presence of unseen items, positioning DenseRec as a practical and robust solution for cold-start sequential recommendation.