# Rank List Sensitivity of Recommender Systems to Interaction Perturbations

arXiv: https://arxiv.org/abs/2201.12686 · HTML: https://mlsanity.com/paper/2201.12686/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 84 references that are in the map.
- First posted on arXiv 29 Jan 2022.

## Builds on (most cited first, 30 of 42)

- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Estimating Training Data Influence by Tracing Gradient Descent](https://mlsanity.com/paper/2002.08484/) (2020)
- [Underspecification Presents Challenges for Credibility in Modern Machine Learning](https://mlsanity.com/paper/2011.03395/) (2020)
- [Predicting Dynamic Embedding Trajectory in Temporal Interaction Networks](https://mlsanity.com/paper/1908.01207/) (2019)
- [KGAT: Knowledge Graph Attention Network for Recommendation](https://mlsanity.com/paper/1905.07854/) (2019)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Certified Defenses for Data Poisoning Attacks](https://mlsanity.com/paper/1706.03691/) (2017)
- [Simple Black-box Adversarial Attacks](https://mlsanity.com/paper/1905.07121/) (2019)
- [Predictive Multiplicity in Classification](https://mlsanity.com/paper/1909.06677/) (2019)
- [Improving the Robustness of Deep Neural Networks via Stability Training](https://mlsanity.com/paper/1604.04326/) (2016)
- [Evaluating the Robustness of Neural Language Models to Input Perturbations](https://mlsanity.com/paper/2108.12237/) (2021)
- [Data Poisoning Attacks on Factorization-Based Collaborative Filtering](https://mlsanity.com/paper/1608.08182/) (2016)
- [Community Interaction and Conflict on the Web](https://mlsanity.com/paper/1803.03697/) (2018)
- [On Counterfactual Explanations under Predictive Multiplicity](https://mlsanity.com/paper/2006.13132/) (2020)
- [Interactive Path Reasoning on Graph for Conversational Recommendation](https://mlsanity.com/paper/2007.00194/) (2020)
- [Characterizing Fairness Over the Set of Good Models Under Selective Labels](https://mlsanity.com/paper/2101.00352/) (2021)
- [Influence Function based Data Poisoning Attacks to Top-N Recommender Systems](https://mlsanity.com/paper/2002.08025/) (2020)
- [Poisoning Attacks to Graph-Based Recommender Systems](https://mlsanity.com/paper/1809.04127/) (2018)
- [Data Poisoning Attacks on Stochastic Bandits](https://mlsanity.com/paper/1905.06494/) (2019)
- [Generative Poisoning Attack Method Against Neural Networks](https://mlsanity.com/paper/1703.01340/) (2017)
- [RepeatNet: A Repeat Aware Neural Recommendation Machine for Session-based Recommendation](https://mlsanity.com/paper/1812.02646/) (2018)
- [Exploring Data Splitting Strategies for the Evaluation of Recommendation Models](https://mlsanity.com/paper/2007.13237/) (2020)
- [Data Poisoning Attacks to Deep Learning Based Recommender Systems](https://mlsanity.com/paper/2101.02644/) (2021)
- [Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction](https://mlsanity.com/paper/2109.01165/) (2021)
- [Leave-one-out Unfairness](https://mlsanity.com/paper/2107.10171/) (2021)
- [Dynamic Graph Collaborative Filtering](https://mlsanity.com/paper/2101.02844/) (2021)

## Built on by (most cited first, 20 of 20)

- [Benchmarking Cognitive Biases in Large Language Models as Evaluators](https://mlsanity.com/paper/2309.17012/) (2023)
- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [Training Data Attribution via Approximate Unrolled Differentiation](https://mlsanity.com/paper/2405.12186/) (2024)
- [Does It Look Sequential? An Analysis of Datasets for Evaluation of Sequential Recommendations](https://mlsanity.com/paper/2408.12008/) (2024)
- [An Experimental Study on the Rashomon Effect of Balancing Methods in Imbalanced Classification](https://mlsanity.com/paper/2405.01557/) (2024)
- [Concept -- An Evaluation Protocol on Conversational Recommender Systems with System-centric and User-centric Factors](https://mlsanity.com/paper/2404.03304/) (2024)
- [Algorithmic Collective Action in Recommender Systems: Promoting Songs by Reordering Playlists](https://mlsanity.com/paper/2404.04269/) (2024)
- [Integrating Item Relevance in Training Loss for Sequential Recommender Systems](https://mlsanity.com/paper/2305.10824/) (2023)
- [Stability and Multigroup Fairness in Ranking with Uncertain Predictions](https://mlsanity.com/paper/2402.09326/) (2024)
- [Towards Adversarially Robust Recommendation from Adaptive Fraudster Detection](https://mlsanity.com/paper/2211.11534/) (2022)
- [Empowering Interdisciplinary Insights with Dynamic Graph Embedding Trajectories](https://mlsanity.com/paper/2406.17963/) (2024)
- [Assumption-free stability for ranking problems](https://mlsanity.com/paper/2506.02257/) (2025)
- [Towards More Robust and Accurate Sequential Recommendation with Cascade-guided Adversarial Training](https://mlsanity.com/paper/2304.05492/) (2023)
- [Investigating the Robustness of Sequential Recommender Systems Against Training Data Perturbations](https://mlsanity.com/paper/2307.13165/) (2023)
- [FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuning](https://mlsanity.com/paper/2402.03481/) (2024)
- [IntentRec: Predicting User Session Intent with Hierarchical Multi-Task Learning](https://mlsanity.com/paper/2408.05353/) (2024)
- [Regretful Decisions under Label Noise](https://mlsanity.com/paper/2504.09330/) (2025)
- [Securing Recommender System via Cooperative Training](https://mlsanity.com/paper/2401.12700/) (2024)
- [Adapting Dual-encoder Vision-language Models for Paraphrased Retrieval](https://mlsanity.com/paper/2405.03190/) (2024)
- [Decoupled Learning and Selection in Slate Recommendation for Privacy and Stability Under Noisy Scores](https://mlsanity.com/paper/2609.29453/) (2026)

## Abstract (arXiv)

Prediction models can exhibit sensitivity with respect to training data: small changes in the training data can produce models that assign conflicting predictions to individual data points during test time. In this work, we study this sensitivity in recommender systems, where users' recommendations are drastically altered by minor perturbations in other unrelated users' interactions. We introduce a measure of stability for recommender systems, called Rank List Sensitivity (RLS), which measures how rank lists generated by a given recommender system at test time change as a result of a perturbation in the training data. We develop a method, CASPER, which uses cascading effect to identify the minimal and systematical perturbation to induce higher instability in a recommender system. Experiments on four datasets show that recommender models are overly sensitive to minor perturbations introduced randomly or via CASPER - even perturbing one random interaction of one user drastically changes the recommendation lists of all users. Importantly, with CASPER perturbation, the models generate more unstable recommendations for low-accuracy users (i.e., those who receive low-quality recommendations) than high-accuracy ones.