# Attention Calibration for Transformer-based Sequential Recommendation

arXiv: https://arxiv.org/abs/2308.09419 · HTML: https://mlsanity.com/paper/2308.09419/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Aug 2023.

## Built on by (most cited first, 15 of 15)

- [An Attentive Inductive Bias for Sequential Recommendation beyond the Self-Attention](https://mlsanity.com/paper/2312.10325/) (2023)
- [TransGNN: Harnessing the Collaborative Power of Transformers and Graph Neural Networks for Recommender Systems](https://mlsanity.com/paper/2308.14355/) (2023)
- [FineRec:Exploring Fine-grained Sequential Recommendation](https://mlsanity.com/paper/2404.12975/) (2024)
- [Disentangling ID and Modality Effects for Session-based Recommendation](https://mlsanity.com/paper/2404.12969/) (2024)
- [CSRec: Rethinking Sequential Recommendation from A Causal Perspective](https://mlsanity.com/paper/2409.05872/) (2024)
- [The End of Reward Engineering: How LLMs Are Redefining Multi-Agent Coordination](https://mlsanity.com/paper/2601.08237/) (2026)
- [No More K-means: Single-Stage Sparse Coding for Efficient Multi-Vector Retrieval](https://mlsanity.com/paper/2605.30120/) (2026)
- [Disentangling Multiplex Spatial-Temporal Transition Graph Representation Learning for Socially Enhanced POI Recommendation](https://mlsanity.com/paper/2508.07649/) (2025)
- [TV-Rec: Time-Variant Convolutional Filter for Sequential Recommendation](https://mlsanity.com/paper/2510.25259/) (2025)
- [Structural and Disentangled Adaptation of Large Vision Language Models for Multimodal Recommendation](https://mlsanity.com/paper/2512.06883/) (2025)
- [Multimodal Enhancement of Sequential Recommendation](https://mlsanity.com/paper/2602.07207/) (2026)
- [Precision Tracked Transformer via Kalman Filtering, Kriging and Process Noise](https://mlsanity.com/paper/2605.18832/) (2026)
- [JourneyFormer: Encoding Airbnb Guest Journey with Sequence Modeling](https://mlsanity.com/paper/2606.19108/) (2026)
- [Sequential Learner Modeling Using Multi-Relational Graph Convolutional Networks](https://mlsanity.com/paper/2607.19253/) (2026)
- [GOD: Enhancing Generalization via Deep Grafting for Sequential Recommendation](https://mlsanity.com/paper/2608.16073/) (2026)

## Abstract (arXiv)

Transformer-based sequential recommendation (SR) has been booming in recent years, with the self-attention mechanism as its key component. Self-attention has been widely believed to be able to effectively select those informative and relevant items from a sequence of interacted items for next-item prediction via learning larger attention weights for these items. However, this may not always be true in reality. Our empirical analysis of some representative Transformer-based SR models reveals that it is not uncommon for large attention weights to be assigned to less relevant items, which can result in inaccurate recommendations. Through further in-depth analysis, we find two factors that may contribute to such inaccurate assignment of attention weights: sub-optimal position encoding and noisy input. To this end, in this paper, we aim to address this significant yet challenging gap in existing works. To be specific, we propose a simple yet effective framework called Attention Calibration for Transformer-based Sequential Recommendation (AC-TSR). In AC-TSR, a novel spatial calibrator and adversarial calibrator are designed respectively to directly calibrates those incorrectly assigned attention weights. The former is devised to explicitly capture the spatial relationships (i.e., order and distance) among items for more precise calculation of attention weights. The latter aims to redistribute the attention weights based on each item's contribution to the next-item prediction. AC-TSR is readily adaptable and can be seamlessly integrated into various existing transformer-based SR models. Extensive experimental results on four benchmark real-world datasets demonstrate the superiority of our proposed ACTSR via significant recommendation performance enhancements. The source code is available at https://github.com/AIM-SE/AC-TSR.