mlsanityData to 7 Oct 2026

Thieves on Sesame Street! Model Extraction of BERT-based APIs

arXiv 1910.12366 ↗First posted 27 Oct 2019cs.CL cs.CR cs.LGcited by 136

Not on the map. Why?

Cited by 22 in LLM safety and security, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We study the problem of model extraction in natural language processing, in which an adversary with only query access to a victim model attempts to reconstruct a local copy of that model. Assuming that both the adversary and victim model fine-tune a large pretrained language model such as BERT (Devlin et al. 2019), we show that the adversary does not need any real training data to successfully mount the attack. In fact, the attacker need not even use grammatical or semantically meaningful queries: we show that random sequences of words coupled with task-specific heuristics form effective queries for model extraction on a diverse set of NLP tasks, including natural language inference and question answering. Our work thus highlights an exploit only made feasible by the shift towards transfer learning methods within the NLP community: for a query budget of a few hundred dollars, an attacker can extract a model that performs only slightly worse than the victim model. Finally, we study two defense strategies against model extraction---membership classification and API watermarking---which while successful against naive adversaries, are ineffective against more sophisticated ones.

Key facts. Cited by 136 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Oct 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 25; 2023: 27; 2024: 25; 2025: 16; 2026: 17.

136 papers.

Sort:

By area:

  1. 2024Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothingcited by 341
  2. 2023The False Promise of Imitating Proprietary LLMscited by 254
  3. 2025Large Language Model Agent: A Survey on Methodology, Applications and Challengescited by 194
  4. 2022Compositional Semantic Parsing with Large Language Modelscited by 95
  5. 2022Membership Inference Attacks by Exploiting Loss Trajectorycited by 93
  6. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 81
  7. 2021ML-Doctor: Holistic Risk Assessment of Inference Attacks Against Machine Learning Modelscited by 79
  8. 2021Protecting Intellectual Property of Language Generation APIs with Lexical Watermarkcited by 78
  9. 2022I Know What You Trained Last Summer: A Survey on Stealing Machine Learning Models and Defencescited by 75
  10. 2023Unleashing the potential of prompt engineering for large language modelscited by 72
  11. 2022Black-box Prompt Learning for Pre-trained Language Modelscited by 70
  12. 2021SoK: How Robust is Image Classification Deep Neural Network Watermarking? (Extended Version)cited by 67
38 more
  1. 2023Language Model Inversioncited by 66
  2. 2024Instructional Fingerprinting of Large Language Modelscited by 64
  3. 2021Mitigating Membership Inference Attacks by Self-Distillation Through a Novel Ensemble Architecturecited by 63
  4. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  5. 2023Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermarkcited by 61
  6. 2022Towards Data-Free Model Stealing in a Hard Label Settingcited by 57
  7. 2021Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!cited by 51
  8. 2021Model Stealing Attacks Against Inductive Graph Neural Networkscited by 50
  9. 2022BITE: Textual Backdoor Attacks with Iterative Trigger Injectioncited by 47
  10. 2021BERT & Family Eat Word Salad: Experiments with Text Understandingcited by 44
  11. 2021Survey: Leakage and Privacy at Inference Timecited by 44
  12. 2022SSLGuard: A Watermarking Scheme for Self-supervised Learning Pre-trained Encoderscited by 43
  13. 2021Knowledge Inheritance for Pre-trained Language Modelscited by 38
  14. 2021Dynamic Knowledge Distillation for Pre-trained Language Modelscited by 38
  15. 2024Logits of API-Protected LLMs Leak Proprietary Informationcited by 38
  16. 2022Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Modelscited by 37
  17. 2021Quantifying and Mitigating Privacy Risks of Contrastive Learningcited by 35
  18. 2021Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safetycited by 35
  19. 2021Black-Box Attacks on Sequential Recommenders via Data-Free Model Extractioncited by 34
  20. 2023A Survey on ChatGPT: AI-Generated Contents, Challenges, and Solutionscited by 34
  21. 2024Tokenization Matters! Degrading Large Language Models through Challenging Their Tokenizationcited by 32
  22. 2022DualCF: Efficient Model Extraction Attack from Counterfactual Explanationscited by 31
  23. 2022Can't Steal? Cont-Steal! Contrastive Stealing Attacks Against Image Encoderscited by 30
  24. 2022Bilateral Dependency Optimization: Defending Against Model-inversion Attackscited by 30
  25. 2023Identifying and Mitigating Privacy Risks Stemming from Language Models: A Surveycited by 30
  26. 2022Distillation-Resistant Watermarking for Model Protection in NLPcited by 29
  27. 2024Large Model Based Agents: State-of-the-Art, Cooperation Paradigms, Security and Privacy, and Future Trendscited by 26
  28. 2024Transferable Embedding Inversion Attack: Uncovering Privacy Risks in Text Embeddings without Model Queriescited by 25
  29. 2023Model Leeching: An Extraction Attack Targeting LLMscited by 22
  30. 2022StolenEncoder: Stealing Pre-trained Encoders in Self-supervised Learningcited by 21
  31. 2022Increasing the Cost of Model Extraction with Calibrated Proof of Workcited by 21
  32. 2022How to Steer Your Adversary: Targeted and Efficient Model Stealing Defenses with Gradient Redirectioncited by 21
  33. 2023Model Extraction Attacks Revisitedcited by 21
  34. 2021Do Question Answering Modeling Improvements Hold Across Benchmarks?cited by 20
  35. 2023Stealing the Decoding Algorithms of Language Modelscited by 19
  36. 2025A Survey on Model Extraction Attacks and Defenses for Large Language Modelscited by 18
  37. 2023False Claims against Model Ownership Resolutioncited by 17
  38. 2023Recyclable Tuning for Continual Pre-trainingcited by 17

50 most cited of 136 All 136 (JSON)