# Gotta Catch 'Em All: Using Honeypots to Catch Adversarial Attacks on Neural Networks

arXiv: https://arxiv.org/abs/1904.08554 · HTML: https://mlsanity.com/paper/1904.08554/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2019.

## Built on by (most cited first, 30 of 40)

- [Glaze: Protecting Artists from Style Mimicry by Text-to-Image Models](https://mlsanity.com/paper/2302.04222/) (2023)
- [Hidden Backdoors in Human-Centric Language Models](https://mlsanity.com/paper/2105.00164/) (2021)
- [Nightshade: Prompt-Specific Poisoning Attacks on Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13828/) (2023)
- [BACKDOORL: Backdoor Attack against Competitive Reinforcement Learning](https://mlsanity.com/paper/2105.00579/) (2021)
- [Handcrafted Backdoors in Deep Neural Networks](https://mlsanity.com/paper/2106.04690/) (2021)
- ["Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice](https://mlsanity.com/paper/2212.14315/) (2022)
- [Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networks](https://mlsanity.com/paper/2110.06904/) (2021)
- [Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent](https://mlsanity.com/paper/2106.15023/) (2021)
- [Quantization Backdoors to Deep Learning Commercial Frameworks](https://mlsanity.com/paper/2108.09187/) (2021)
- [A LLM Assisted Exploitation of AI-Guardian](https://mlsanity.com/paper/2307.15008/) (2023)
- [AutoAdvExBench: Benchmarking autonomous exploitation of adversarial example defenses](https://mlsanity.com/paper/2503.01811/) (2025)
- [Attack as Defense: Characterizing Adversarial Examples using Robustness](https://mlsanity.com/paper/2103.07633/) (2021)
- [Increasing Confidence in Adversarial Robustness Evaluations](https://mlsanity.com/paper/2206.13991/) (2022)
- [What You See is Not What the Network Infers: Detecting Adversarial Examples Based on Semantic Contradiction](https://mlsanity.com/paper/2201.09650/) (2022)
- [Progressive Backdoor Erasing via connecting Backdoor and Adversarial Attacks](https://mlsanity.com/paper/2202.06312/) (2022)
- [Towards Unified Robustness Against Both Backdoor and Adversarial Attacks](https://mlsanity.com/paper/2405.17929/) (2024)
- [Evaluating the Robustness of Trigger Set-Based Watermarks Embedded in Deep Neural Networks](https://mlsanity.com/paper/2106.10147/) (2021)
- [A Tale of Two Models: Constructing Evasive Attacks on Edge Models](https://mlsanity.com/paper/2204.10933/) (2022)
- [Be Your Own Neighborhood: Detecting Adversarial Example by the Neighborhood Relations Built on Self-Supervised Learning](https://mlsanity.com/paper/2209.00005/) (2022)
- [Trap-MID: Trapdoor-based Defense against Model Inversion Attacks](https://mlsanity.com/paper/2411.08460/) (2024)
- [Transpose Attack: Stealing Datasets with Bidirectional Training](https://mlsanity.com/paper/2311.07389/) (2023)
- [Fortify the Guardian, Not the Treasure: Resilient Adversarial Detectors](https://mlsanity.com/paper/2404.12120/) (2024)
- [RBNN: Memory-Efficient Reconfigurable Deep Binary Neural Network with IP Protection for Internet of Things](https://mlsanity.com/paper/2105.03822/) (2021)
- [Let Them Steal: Trapping Large Language Model Extraction Attacks with Knowledge Honeypot](https://mlsanity.com/paper/2606.15810/) (2026)
- ["What's in the box?!": Deflecting Adversarial Attacks by Randomly Deploying Adversarially-Disjoint Models](https://mlsanity.com/paper/2102.05104/) (2021)
- [TRAPDOOR: Repurposing backdoors to detect dataset bias in machine learning-based genomic analysis](https://mlsanity.com/paper/2108.10132/) (2021)
- [SeqNet: An Efficient Neural Network for Automatic Malware Detection](https://mlsanity.com/paper/2205.03850/) (2022)
- [Fight Fire with Fire: Combating Adversarial Patch Attacks using Pattern-randomized Defensive Patches](https://mlsanity.com/paper/2311.06122/) (2023)
- [Evolutionary Trigger Detection and Lightweight Model Repair Based Backdoor Defense](https://mlsanity.com/paper/2407.05396/) (2024)
- [Revocable Backdoor for Deep Model Trading](https://mlsanity.com/paper/2408.00255/) (2024)

## Abstract (arXiv)

Deep neural networks (DNN) are known to be vulnerable to adversarial attacks. Numerous efforts either try to patch weaknesses in trained models, or try to make it difficult or costly to compute adversarial examples that exploit them. In our work, we explore a new "honeypot" approach to protect DNN models. We intentionally inject trapdoors, honeypot weaknesses in the classification manifold that attract attackers searching for adversarial examples. Attackers' optimization algorithms gravitate towards trapdoors, leading them to produce attacks similar to trapdoors in the feature space. Our defense then identifies attacks by comparing neuron activation signatures of inputs to those of trapdoors. In this paper, we introduce trapdoors and describe an implementation of a trapdoor-enabled defense. First, we analytically prove that trapdoors shape the computation of adversarial attacks so that attack inputs will have feature representations very similar to those of trapdoors. Second, we experimentally show that trapdoor-protected models can detect, with high accuracy, adversarial examples generated by state-of-the-art attacks (PGD, optimization-based CW, Elastic Net, BPDA), with negligible impact on normal classification. These results generalize across classification domains, including image, facial, and traffic-sign recognition. We also present significant results measuring trapdoors' robustness against customized adaptive attacks (countermeasures).