# The Limits of Query Monitoring against Distributed Model Extraction

arXiv: https://arxiv.org/abs/2610.13083 · HTML: https://mlsanity.com/paper/2610.13083/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 36 of its 91 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Builds on (most cited first, 30 of 36)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers](https://mlsanity.com/paper/2002.10957/) (2020)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [Stealing Machine Learning Models via Prediction APIs](https://mlsanity.com/paper/1609.02943/) (2016)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [Knockoff Nets: Stealing Functionality of Black-Box Models](https://mlsanity.com/paper/1812.02766/) (2018)
- [The False Promise of Imitating Proprietary LLMs](https://mlsanity.com/paper/2305.15717/) (2023)
- [Magicoder: Empowering Code Generation with OSS-Instruct](https://mlsanity.com/paper/2312.02120/) (2023)
- [PRADA: Protecting against DNN Model Stealing Attacks](https://mlsanity.com/paper/1805.02628/) (2018)
- [Game-theoretic statistics and safe anytime-valid inference](https://mlsanity.com/paper/2210.01948/) (2022)
- [DAWN: Dynamic Adversarial Watermarking of Neural Networks](https://mlsanity.com/paper/1906.00830/) (2019)
- [Spectral redemption: clustering sparse networks](https://mlsanity.com/paper/1306.5550/) (2013)
- [Stateful Detection of Black-Box Adversarial Attacks](https://mlsanity.com/paper/1907.05587/) (2019)
- [Identifying Coordinated Accounts on Social Media through Hidden Influence and Group Behaviours](https://mlsanity.com/paper/2008.11308/) (2020)
- [Model Extraction Warning in MLaaS Paradigm](https://mlsanity.com/paper/1711.07221/) (2017)
- [Privacy Side Channels in Machine Learning Systems](https://mlsanity.com/paper/2309.05610/) (2023)
- [Design and Analysis of Switchback Experiments](https://mlsanity.com/paper/2009.00148/) (2020)
- [Blacklight: Scalable Defense for Neural Networks against Query-Based Black-Box Attacks](https://mlsanity.com/paper/2006.14042/) (2020)
- [Heteroskedastic PCA: Algorithm, Optimality, and Applications](https://mlsanity.com/paper/1810.08316/) (2018)
- [Testing randomness](https://mlsanity.com/paper/1906.09256/) (2019)
- [Retrain or not retrain: Conformal test martingales for change-point detection](https://mlsanity.com/paper/2102.10439/) (2021)
- [Concentration and regularization of random graphs](https://mlsanity.com/paper/1506.00669/) (2015)
- [Spectral Clustering of Graphs with the Bethe Hessian](https://mlsanity.com/paper/1406.1880/) (2014)
- [Computational Lower Bounds for Community Detection on Random Graphs](https://mlsanity.com/paper/1406.6625/) (2014)
- [On the Detection of Disinformation Campaign Activity with Network Analysis](https://mlsanity.com/paper/2005.13466/) (2020)

## Abstract (arXiv)

Model providers now face distillation campaigns run through thousands of accounts, each of which looks like an ordinary user. When network and payment metadata cannot link those accounts, the only evidence left is what they ask. We study what a provider can and cannot learn from those queries alone, using one million real chat turns for development and 17 later weeks that no design choice touched for a single frozen evaluation. The answer is set less by anything specific to extraction than by benign coordination: a campaign is visible only if it is more distinctive than the most distinctive group of ordinary users in the same week. We make this precise, show that a small but positive difference between campaign and users buys a detection time rather than detectability, and identify the campaign that is hardest to detect. Empirically, a calibrated monitor catches naive distributed extraction that per-account monitors miss (0.86-1.00 of weeks), and one variant - selecting accounts that move together after discounting how topical each one is - still catches campaigns padded with natural text (0.50-0.96 of weeks) where the other two spectral detectors catch none. Distilling each campaign into a small student shows that, in our testbed, campaigns that evaded every detector kept at most 42% of the capability of a detectable one. The limits are equally concrete: a campaign that ramps up slowly poisons every detector's calibration, benign evaluation pipelines look exactly like distillation, we found no deployable detector at colluder shares of 2.5% or below, and campaigns that relay real users' requests may be invisible to any query-only monitor.