# Investigating the Robustness of Sequential Recommender Systems Against Training Data Perturbations

arXiv: https://arxiv.org/abs/2307.13165 · HTML: https://mlsanity.com/paper/2307.13165/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 54 references that are in the map.
- First posted on arXiv 24 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Neural Graph Collaborative Filtering](https://mlsanity.com/paper/1905.08108/) (2019)
- [Graph Neural Networks in Recommender Systems: A Survey](https://mlsanity.com/paper/2011.02260/) (2020)
- [Fairness of Exposure in Rankings](https://mlsanity.com/paper/1802.07281/) (2018)
- [RecBole: Towards a Unified, Comprehensive and Efficient Framework for Recommendation Algorithms](https://mlsanity.com/paper/2011.01731/) (2020)
- [Sequential Recommender Systems: Challenges, Progress and Prospects](https://mlsanity.com/paper/2001.04830/) (2019)
- [Robustness Gym: Unifying the NLP Evaluation Landscape](https://mlsanity.com/paper/2101.04840/) (2021)
- [Sequential Recommendation with Graph Neural Networks](https://mlsanity.com/paper/2106.14226/) (2021)
- [Personalizing Session-based Recommendations with Hierarchical Recurrent Neural Networks](https://mlsanity.com/paper/1706.04148/) (2017)
- [User-oriented Fairness in Recommendation](https://mlsanity.com/paper/2104.10671/) (2021)
- [Continuous-Time Sequential Recommendation with Temporal Graph Collaborative Transformer](https://mlsanity.com/paper/2108.06625/) (2021)
- [Sequence-Aware Recommender Systems](https://mlsanity.com/paper/1802.08452/) (2018)
- [Current Challenges and Visions in Music Recommender Systems Research](https://mlsanity.com/paper/1710.03208/) (2017)
- [Rank List Sensitivity of Recommender Systems to Interaction Perturbations](https://mlsanity.com/paper/2201.12686/) (2022)
- [Explainability in Music Recommender Systems](https://mlsanity.com/paper/2201.10528/) (2022)
- [Effective and Efficient Training for Sequential Recommendation using Recency Sampling](https://mlsanity.com/paper/2207.02643/) (2022)
- [Improving Training Stability for Multitask Ranking Models in Recommender Systems](https://mlsanity.com/paper/2302.09178/) (2023)
- [RGRecSys: A Toolkit for Robustness Evaluation of Recommender Systems](https://mlsanity.com/paper/2201.04399/) (2022)
- [Sheaf4Rec: Sheaf Neural Networks for Graph-based Recommender Systems](https://mlsanity.com/paper/2304.09097/) (2023)
- [Learning with Noisy Labels through Learnable Weighting and Centroid Similarity](https://mlsanity.com/paper/2303.09470/) (2023)
- [Integrating Item Relevance in Training Loss for Sequential Recommender Systems](https://mlsanity.com/paper/2305.10824/) (2023)

## Built on by (most cited first, 2 of 2)

- [Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendations](https://mlsanity.com/paper/2408.12008/) (2024)
- [FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuning](https://mlsanity.com/paper/2402.03481/) (2024)

## Abstract (arXiv)

Sequential Recommender Systems (SRSs) are widely employed to model user behavior over time. However, their robustness in the face of perturbations in training data remains a largely understudied yet critical issue. A fundamental challenge emerges in previous studies aimed at assessing the robustness of SRSs: the Rank-Biased Overlap (RBO) similarity is not particularly suited for this task as it is designed for infinite rankings of items and thus shows limitations in real-world scenarios. For instance, it fails to achieve a perfect score of 1 for two identical finite-length rankings. To address this challenge, we introduce a novel contribution: Finite Rank-Biased Overlap (FRBO), an enhanced similarity tailored explicitly for finite rankings. This innovation facilitates a more intuitive evaluation in practical settings. In pursuit of our goal, we empirically investigate the impact of removing items at different positions within a temporally ordered sequence. We evaluate two distinct SRS models across multiple datasets, measuring their performance using metrics such as Normalized Discounted Cumulative Gain (NDCG) and Rank List Sensitivity. Our results demonstrate that removing items at the end of the sequence has a statistically significant impact on performance, with NDCG decreasing up to 60%. Conversely, removing items from the beginning or middle has no significant effect. These findings underscore the criticality of the position of perturbed items in the training data. As we spotlight the vulnerabilities inherent in current SRSs, we fervently advocate for intensified research efforts to fortify their robustness against adversarial perturbations.