# Stateful Detection of Model Extraction Attacks

arXiv: https://arxiv.org/abs/2107.05166 · HTML: https://mlsanity.com/paper/2107.05166/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 12 Jul 2021.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Adversarial examples in the physical world](https://mlsanity.com/paper/1607.02533/) (2016)
- [DeepFool: a simple and accurate method to fool deep neural networks](https://mlsanity.com/paper/1511.04599/) (2015)
- [Practical Black-Box Attacks against Machine Learning](https://mlsanity.com/paper/1602.02697/) (2016)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods](https://mlsanity.com/paper/1705.07263/) (2017)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Models](https://mlsanity.com/paper/1805.06605/) (2018)
- [Deep Learning for Anomaly Detection: A Survey](https://mlsanity.com/paper/1901.03407/) (2019)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [MagNet: a Two-Pronged Defense against Adversarial Examples](https://mlsanity.com/paper/1705.09064/) (2017)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Adversarial Active Learning for Deep Networks: a Margin Based Approach](https://mlsanity.com/paper/1802.09841/) (2018)
- [Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Data](https://mlsanity.com/paper/1806.05476/) (2018)
- [Model Extraction Warning in MLaaS Paradigm](https://mlsanity.com/paper/1711.07221/) (2017)
- [Extraction of Complex DNN Models: Real Threat or Boogeyman?](https://mlsanity.com/paper/1910.05429/) (2019)
- [Defending Against Adversarial Attacks by Leveraging an Entire GAN](https://mlsanity.com/paper/1805.10652/) (2018)

## Built on by (most cited first, 8 of 8)

- [I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defences](https://mlsanity.com/paper/2206.08451/) (2022)
- [Privacy Side Channels in Machine Learning Systems](https://mlsanity.com/paper/2309.05610/) (2023)
- [Increasing the Cost of Model Extraction with Calibrated Proof of Work](https://mlsanity.com/paper/2201.09243/) (2022)
- [Marich: A Query-efficient Distributionally Equivalent Model Extraction Attack using Public Data](https://mlsanity.com/paper/2302.08466/) (2023)
- [MalProtect: Stateful Defense Against Adversarial Query Attacks in ML-based Malware Detection](https://mlsanity.com/paper/2302.10739/) (2023)
- [ATOM: A Framework of Detecting Query-Based Model Extraction Attacks for Graph Neural Networks](https://mlsanity.com/paper/2503.16693/) (2025)
- [FDINet: Protecting against DNN Model Extraction via Feature Distortion Index](https://mlsanity.com/paper/2306.11338/) (2023)
- [Pareto-Secure Machine Learning (PSML): Fingerprinting and Securing Inference Serving Systems](https://mlsanity.com/paper/2307.01292/) (2023)

## Abstract (arXiv)

Machine-Learning-as-a-Service providers expose machine learning (ML) models through application programming interfaces (APIs) to developers. Recent work has shown that attackers can exploit these APIs to extract good approximations of such ML models, by querying them with samples of their choosing. We propose VarDetect, a stateful monitor that tracks the distribution of queries made by users of such a service, to detect model extraction attacks. Harnessing the latent distributions learned by a modified variational autoencoder, VarDetect robustly separates three types of attacker samples from benign samples, and successfully raises an alarm for each. Further, with VarDetect deployed as an automated defense mechanism, the extracted substitute models are found to exhibit poor performance and transferability, as intended. Finally, we demonstrate that even adaptive attackers with prior knowledge of the deployment of VarDetect, are detected by it.