mlsanityData to 14 Oct 2026

Let Them Steal: Trapping Large Language Model Extraction Attacks with Knowledge Honeypot

arXiv 2606.15810 ↗First posted 14 Jun 2026cs.CR cs.AIcited by 3

Not on the map. Why?

Abstract (arXiv)

Large language models deployed as commercial APIs are vulnerable to model extraction attacks, while existing defenses either act too late or degrade utility for legitimate users. We propose Knowledge Trap, a defense that redirects extraction attacks toward low-transferability knowledge through a Honeypot Knowledge Graph (HKG) and breadcrumb-guided exploration. Instead of blocking queries or perturbing outputs, Knowledge Trap consumes the attacker's limited query budget on knowledge with negligible downstream utility while preserving benign-user performance. Experiments in medical and financial domains show that Knowledge Trap reduces surrogate Agreement by 6.2% on average without degrading legitimate-user accuracy, outperforming existing defenses that impose measurable user impact. These results suggest that defending knowledge-space traversal is a practical direction for mitigating LLM extraction attacks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 42 of its 61 references that are in the map. First posted on arXiv 14 Jun 2026.

Builds on

42 of 61 references.

References · 42

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  3. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  4. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  5. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  6. 2021Membership Inference Attacks From First Principlescited by 878
  7. 2020A Survey on Knowledge Graphs: Representation, Acquisition and Applicationscited by 643
  8. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  9. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 562
  10. 2021QA-GNN: Reasoning with Language Models and Knowledge Graphs for Question Answeringcited by 388
  11. 2019K-BERT: Enabling Language Representation with Knowledge Graphcited by 304
  12. 2018Knockoff Nets: Stealing Functionality of Black-Box Modelscited by 292
30 more
  1. 2023Think-on-Graph: Deep and Responsible Reasoning of Large Language Model on Knowledge Graphcited by 238
  2. 2019High Accuracy and High Fidelity Extraction of Neural Networkscited by 214
  3. 2021When Does Pretraining Help? Assessing Self-Supervised Learning for Law and the CaseHOLD Datasetcited by 201
  4. 2022ConvFinQA: Exploring the Chain of Numerical Reasoning in Conversational Finance Question Answeringcited by 196
  5. 2022GreaseLM: Graph REASoning Enhanced Language Models for Question Answeringcited by 171
  6. 2018PRADA: Protecting against DNN Model Stealing Attackscited by 165
  7. 2024Stealing Part of a Production Language Modelcited by 159
  8. 2020Entangled Watermarks as a Defense against Model Extractioncited by 149
  9. 2019Thieves on Sesame Street! Model Extraction of BERT-based APIscited by 139
  10. 2019DAWN: Dynamic Adversarial Watermarking of Neural Networkscited by 100
  11. 2020CoLAKE: Contextualized Language and Knowledge Embeddingcited by 86
  12. 2018Exploring Connections Between Active Learning and Model Extractioncited by 85
  13. 2019Prediction Poisoning: Towards Defenses Against DNN Model Stealing Attackscited by 81
  14. 2022CATER: Intellectual Property Protection on Text Generation APIs via Conditional Watermarkscited by 76
  15. 2018Copycat CNN: Stealing Knowledge by Persuading Confession with Random Non-Labeled Datacited by 70
  16. 2023Are You Copying My Model? Protecting the Copyright of Large Language Models for EaaS via Backdoor Watermarkcited by 67
  17. 2019Defending Against Model Stealing Attacks with Adaptive Misinformationcited by 57
  18. 2019Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networkscited by 40
  19. 2022Distillation-Resistant Watermarking for Model Protection in NLPcited by 30
  20. 2017A Game-Theoretic Taxonomy and Survey of Defensive Deception for Cybersecurity and Privacycited by 22
  21. 2023Model Leeching: An Extraction Attack Targeting LLMscited by 22
  22. 2022Increasing the Cost of Model Extraction with Calibrated Proof of Workcited by 21
  23. 2021Student Surpasses Teacher: Imitation Attack for Black-Box NLP APIscited by 18
  24. 2020A Sweet Rabbit Hole by DARCY: Using Honeypots to Detect Universal Trigger's Adversarial Attackscited by 15
  25. 2024LLM Honeypot: Leveraging Large Language Models as Advanced Interactive Honeypot Systemscited by 10
  26. 2025MISLEADER: Defending against Model Extraction with Ensembles of Distilled Modelscited by 8
  27. 2023MeaeQ: Mount Model Extraction Attacks with Efficient Queriescited by 6
  28. 2025HoneypotNet: Backdoor Attacks Against Model Extractioncited by 6
  29. 2024NSmark: Null Space Based Black-box Watermarking Defense Framework for Language Modelscited by 4
  30. 2026GraphIP-Bench: How Hard Is It to Steal a Graph Neural Network, and Can We Stop It?cited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026HoneyRoute: Honeypot-Model Routing for Adversarial LLM Servingcited by 0
  2. 2026Capability Scaling-Down Laws for LLM Compressioncited by 0
  3. 2026The Limits of Query Monitoring against Distributed Model Extractioncited by 0