# Black-Box Attacks on Sequential Recommenders via Data-Free Model Extraction

arXiv: https://arxiv.org/abs/2109.01165 · HTML: https://mlsanity.com/paper/2109.01165/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 47 references that are in the map.
- First posted on arXiv 1 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [Self-Attentive Sequential Recommendation](https://mlsanity.com/paper/1808.09781/) (2018)
- [Neural Collaborative Filtering](https://mlsanity.com/paper/1708.05031/) (2017)
- [BERT4Rec: Sequential Recommendation with Bidirectional Encoder Representations from Transformer](https://mlsanity.com/paper/1904.06690/) (2019)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Image-based Recommendations on Styles and Substitutes](https://mlsanity.com/paper/1506.04757/) (2015)
- [Session-based Recommendations with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06939/) (2015)
- [Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment](https://mlsanity.com/paper/1907.11932/) (2019)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding](https://mlsanity.com/paper/1809.07426/) (2018)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [Neural Attentive Session-based Recommendation](https://mlsanity.com/paper/1711.04725/) (2017)
- [Fusing Similarity Models with Markov Chains for Sparse Sequential Recommendation](https://mlsanity.com/paper/1609.09152/) (2016)
- [Thieves on Sesame Street! Model Extraction of BERT-based APIs](https://mlsanity.com/paper/1910.12366/) (2019)
- [MAZE: Data-Free Model Stealing Attack Using Zeroth-Order Gradient Estimation](https://mlsanity.com/paper/2005.03161/) (2020)
- [Data Poisoning Attacks on Factorization-Based Collaborative Filtering](https://mlsanity.com/paper/1608.08182/) (2016)
- [Influence Function based Data Poisoning Attacks to Top-N Recommender Systems](https://mlsanity.com/paper/2002.08025/) (2020)
- [DaST: Data-free Substitute Training for Adversarial Attacks](https://mlsanity.com/paper/2003.12703/) (2020)
- [Data Poisoning Attacks to Deep Learning Based Recommender Systems](https://mlsanity.com/paper/2101.02644/) (2021)
- [Deep Reinforcement Learning for List-wise Recommendations](https://mlsanity.com/paper/1801.00209/) (2017)
- [Revisiting Adversarially Learned Injection Attacks Against Recommender Systems](https://mlsanity.com/paper/2008.04876/) (2020)
- [Practical Data Poisoning Attack against Next-Item Recommendation](https://mlsanity.com/paper/2004.03728/) (2020)
- [A framework for the extraction of Deep Neural Networks by leveraging public data](https://mlsanity.com/paper/1905.09165/) (2019)

## Built on by (most cited first, 30 of 37)

- [Filter-enhanced MLP is All You Need for Sequential Recommendation](https://mlsanity.com/paper/2202.13556/) (2022)
- [I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences](https://mlsanity.com/paper/2206.08451/) (2022)
- [A Review of Modern Recommender Systems Using Generative Models (Gen-RecSys)](https://mlsanity.com/paper/2404.00579/) (2024)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [Data-Free Knowledge Transfer: A Survey](https://mlsanity.com/paper/2112.15278/) (2021)
- [Knowledge-enhanced Black-box Attacks for Recommendations](https://mlsanity.com/paper/2207.10307/) (2022)
- [A Systematic Review and Replicability Study of BERT4Rec for Sequential Recommendation](https://mlsanity.com/paper/2207.07483/) (2022)
- [Rank List Sensitivity of Recommender Systems to Interaction Perturbations](https://mlsanity.com/paper/2201.12686/) (2022)
- [Dual Student Networks for Data-Free Model Stealing](https://mlsanity.com/paper/2309.10058/) (2023)
- [Recent Developments in Recommender Systems: A Survey](https://mlsanity.com/paper/2306.12680/) (2023)
- [Manipulating Recommender Systems: A Survey of Poisoning Attacks and Countermeasures](https://mlsanity.com/paper/2404.14942/) (2024)
- [WARDEN: Multi-Directional Backdoor Watermarks for Embedding-as-a-Service Copyright Protection](https://mlsanity.com/paper/2403.01472/) (2024)
- [Defending Substitution-Based Profile Pollution Attacks on Sequential Recommenders](https://mlsanity.com/paper/2207.11237/) (2022)
- [Shilling Black-box Review-based Recommender Systems through Fake Review Generation](https://mlsanity.com/paper/2306.16526/) (2023)
- [Algorithmic Collective Action in Recommender Systems: Promoting Songs by Reordering Playlists](https://mlsanity.com/paper/2404.04269/) (2024)
- [Practical Cross-System Shilling Attacks with Limited Access to Data](https://mlsanity.com/paper/2302.07145/) (2023)
- [Fake Resume Attacks: Data Poisoning on Online Job Platforms](https://mlsanity.com/paper/2402.14124/) (2024)
- [Towards More Robust and Accurate Sequential Recommendation with Cascade-guided Adversarial Training](https://mlsanity.com/paper/2304.05492/) (2023)
- [FINEST: Stabilizing Recommendations by Rank-Preserving Fine-Tuning](https://mlsanity.com/paper/2402.03481/) (2024)
- [Privacy in LLM-based Recommendation: Recent Advances and Future Directions](https://mlsanity.com/paper/2406.01363/) (2024)
- [BESA: Boosting Encoder Stealing Attack with Perturbation Recovery](https://mlsanity.com/paper/2506.04556/) (2025)
- [With a Little Help From My Friends: Collective Manipulation in Risk-Controlling Recommender Systems](https://mlsanity.com/paper/2603.28476/) (2026)
- [Latent Code Augmentation Based on Stable Diffusion for Data-free Substitute Attacks](https://mlsanity.com/paper/2307.12872/) (2023)
- [Defense Against Model Extraction Attacks on Recommender Systems](https://mlsanity.com/paper/2310.16335/) (2023)
- [Adversarial Text Rewriting for Text-aware Recommender Systems](https://mlsanity.com/paper/2408.00312/) (2024)
- [ELASTIC: Efficient Linear Attention for Sequential Interest Compression](https://mlsanity.com/paper/2408.09380/) (2024)
- [Obsidian: Cooperative State-Space Exploration for Performant Inference on Secure ML Accelerators](https://mlsanity.com/paper/2409.02817/) (2024)
- [IndirectAD: Practical Data Poisoning Attacks against Recommender Systems for Item Promotion](https://mlsanity.com/paper/2511.05845/) (2025)
- [AI Security in the Foundation Model Era: A Comprehensive Survey from a Unified Perspective](https://mlsanity.com/paper/2603.24857/) (2026)
- [Revealing Secrets From Pre-trained Models](https://mlsanity.com/paper/2207.09539/) (2022)

## Abstract (arXiv)

We investigate whether model extraction can be used to "steal" the weights of sequential recommender systems, and the potential threats posed to victims of such attacks. This type of risk has attracted attention in image and text classification, but to our knowledge not in recommender systems. We argue that sequential recommender systems are subject to unique vulnerabilities due to the specific autoregressive regimes used to train them. Unlike many existing recommender attackers, which assume the dataset used to train the victim model is exposed to attackers, we consider a data-free setting, where training data are not accessible. Under this setting, we propose an API-based model extraction method via limited-budget synthetic data generation and knowledge distillation. We investigate state-of-the-art models for sequential recommendation and show their vulnerability under model extraction and downstream attacks. We perform attacks in two stages. (1) Model extraction: given different types of synthetic data and their labels retrieved from a black-box recommender, we extract the black-box model to a white-box model via distillation. (2) Downstream attacks: we attack the black-box model with adversarial samples generated by the white-box recommender. Experiments show the effectiveness of our data-free model extraction and downstream attacks on sequential recommenders in both profile pollution and data poisoning settings.