# Exploiting Session Information in BERT-based Session-aware Sequential Recommendation

arXiv: https://arxiv.org/abs/2204.10851 · HTML: https://mlsanity.com/paper/2204.10851/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 65 references that are in the map.
- First posted on arXiv 22 Apr 2022.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Session-based Recommendation with Graph Neural Networks](https://mlsanity.com/paper/1811.00855/) (2018)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Hierarchical Gating Networks for Sequential Recommendation](https://mlsanity.com/paper/1906.09217/) (2019)
- [Controllable Multi-Interest Framework for Recommendation](https://mlsanity.com/paper/2005.09347/) (2020)
- [Intent Contrastive Learning for Sequential Recommendation](https://mlsanity.com/paper/2202.02519/) (2022)
- [Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks](https://mlsanity.com/paper/1706.04148/) (2017)
- [Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer](https://mlsanity.com/paper/2108.06625/) (2021)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [Dynamic Graph Neural Networks for Sequential Recommendation](https://mlsanity.com/paper/2104.07368/) (2021)
- [Self-attention with Functional Time Representation Learning](https://mlsanity.com/paper/1911.12864/) (2019)
- [RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation](https://mlsanity.com/paper/1812.02646/) (2018)
- [Sequential Recommendation via Stochastic Self-Attention](https://mlsanity.com/paper/2201.06035/) (2022)
- [Sequential Recommendation with Self-Attentive Multi-Adversarial Network](https://mlsanity.com/paper/2005.10602/) (2020)
- [Lightweight Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/2108.11333/) (2021)
- [MEANTIME: Mixture of Attention Mechanisms with Multi-temporal Embeddings for Sequential Recommendation](https://mlsanity.com/paper/2008.08273/) (2020)
- [Towards Neural Mixture Recommender for Long Range Dependent User Sequences](https://mlsanity.com/paper/1902.08588/) (2019)
- [Session-aware Recommendation: A Surprising Quest for the State-of-the-art](https://mlsanity.com/paper/2011.03424/) (2020)
- [Personalized Graph Neural Networks with Attention Mechanism for Session-Aware Recommendation](https://mlsanity.com/paper/1910.08887/) (2019)
- [Session-aware Information Embedding for E-commerce Product Recommendation](https://mlsanity.com/paper/1707.05955/) (2017)
- [Multi-head Temporal Attention-Augmented Bilinear Network for Financial time series prediction](https://mlsanity.com/paper/2201.05459/) (2022)

## Built on by (most cited first, 4 of 4)

- [Uniform Sequence Better: Time Interval Aware Data Augmentation for Sequential Recommendation](https://mlsanity.com/paper/2212.08262/) (2022)
- [Proxy-based Item Representation for Attribute and Context-aware Recommendation](https://mlsanity.com/paper/2312.06145/) (2023)
- [Not Just What, But When: Integrating Irregular Intervals to LLM for Sequential Recommendation](https://mlsanity.com/paper/2507.23209/) (2025)
- [BERT4Loc: BERT for Location -- POI Recommender System](https://mlsanity.com/paper/2208.01375/) (2022)

## Abstract (arXiv)

In recommendation systems, utilizing the user interaction history as sequential information has resulted in great performance improvement. However, in many online services, user interactions are commonly grouped by sessions that presumably share preferences, which requires a different approach from ordinary sequence representation techniques. To this end, sequence representation models with a hierarchical structure or various viewpoints have been developed but with a rather complex network structure. In this paper, we propose three methods to improve recommendation performance by exploiting session information while minimizing additional parameters in a BERT-based sequential recommendation model: using session tokens, adding session segment embeddings, and a time-aware self-attention. We demonstrate the feasibility of the proposed methods through experiments on widely used recommendation datasets.