mlsanityData to 7 Oct 2026

Student Surpasses Teacher: Imitation Attack for Black-Box NLP APIs

arXiv 2108.13873 ↗First posted 29 Aug 2021cs.CR cs.CL cs.LGcited by 16

Not on the map. Why?

Abstract (arXiv)

Machine-learning-as-a-service (MLaaS) has attracted millions of users to their splendid large-scale models. Although published as black-box APIs, the valuable models behind these services are still vulnerable to imitation attacks. Recently, a series of works have demonstrated that attackers manage to steal or extract the victim models. Nonetheless, none of the previous stolen models can outperform the original black-box APIs. In this work, we conduct unsupervised domain adaptation and multi-victim ensemble to showing that attackers could potentially surpass victims, which is beyond previous understanding of model extraction. Extensive experiments on both benchmark datasets and real-world APIs validate that the imitators can succeed in outperforming the original black-box models on transferred domains. We consider our work as a milestone in the research of imitation attack, especially on NLP APIs, as the superior performance could influence the defense or even publishing strategy of API providers.

Key facts. Cited by 16 papers in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 29 Aug 2021.

Builds on

29 of 45 references.

References · 29

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  6. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,906
  7. 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
  8. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  9. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 762
  10. 2016Stealing Machine Learning Models via Prediction APIscited by 602
  11. 2018Born Again Neural Networkscited by 539
  12. 2013Good Debt or Bad Debt: Detecting Semantic Orientations in Economic Textscited by 310
17 more
  1. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 285
  2. 2020Data Augmentation using Pre-trained Transformer Modelscited by 168
  3. 2019Distilling Task-Specific Knowledge from BERT into Simple Neural Networkscited by 152
  4. 2018Large scale distributed neural network training through online distillationcited by 144
  5. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 136
  6. 2019Classification Accuracy Score for Conditional Generative Modelscited by 134
  7. 2017Learned in Translation: Contextualized Word Vectorscited by 104
  8. 2019Multilingual Neural Machine Translation with Knowledge Distillationcited by 95
  9. 2019BAM! Born-Again Multi-Task Networks for Natural Language Understandingcited by 85
  10. 2020Imitation Attacks and Defenses for Black-box Machine Translation Systemscited by 78
  11. 2019Diversity with Cooperation: Ensemble Methods for Few-Shot Classificationcited by 70
  12. 2020Differentially Private Representation for NLP: Formal Guarantee and An Empirical Study on Privacy and Fairnesscited by 66
  13. 2019This dataset does not exist: training models from generated imagescited by 65
  14. 2021Model Extraction and Adversarial Transferability, Your BERT is Vulnerable!cited by 51
  15. 2016Text Classification Improved by Integrating Bidirectional LSTM with Two-dimensional Max Poolingcited by 39
  16. 2019Simplified Neural Unsupervised Domain Adaptationcited by 10
  17. 2020Collective Wisdom: Improving Low-resource Neural Machine Translation using Adaptive Knowledge Distillationcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 6; 2025: 3; 2026: 2.

16 papers.

Sort:

By area:

  1. 2023When Foundation Model Meets Federated Learning: Motivations, Challenges, and Future Directionscited by 83
  2. 2023Protecting Language Generation Models via Invisible Watermarkingcited by 81
  3. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  4. 2025A Survey on Model Extraction Attacks and Defenses for Large Language Modelscited by 18
  5. 2022IDEAL: Query-Efficient Data-Free Learning from Black-box Modelscited by 16
  6. 2024WARDEN: Multi-Directional Backdoor Watermarks for Embedding-as-a-Service Copyright Protectioncited by 9
  7. 2023MeaeQ: Mount Model Extraction Attacks with Efficient Queriescited by 6
  8. 2024"Yes, My LoRD." Guiding Language Model Extraction with Locality Reinforced Distillationcited by 5
  9. 2024WET: Overcoming Paraphrasing Vulnerabilities in Embeddings-as-a-Service with Linear Transformation Watermarkscited by 5
  10. 2024Generative Models are Self-Watermarked: Declaring Model Authentication through Re-Generationcited by 3
  11. 2025SoK: Are Watermarks in LLMs Ready for Deployment?cited by 2
  12. 2025From Essence to Defense: Adaptive Semantic-aware Watermarking for Embedding-as-a-Service Copyright Protectioncited by 2
4 more
  1. 2026Robust LLM Watermarking with Minimal Semantic Distortion for IP Protectioncited by 1
  2. 2026Let Them Steal: Trapping Large Language Model Extraction Attacks with Knowledge Honeypotcited by 1
  3. 2022Semantics-Preserved Distortion for Personal Privacy Protection in Information Managementcited by 0
  4. 2024Stolen Subwords: Importance of Vocabularies for Machine Translation Model Stealingcited by 0