mlsanityData to 7 Oct 2026

Black-box Detection of Backdoor Attacks with Limited Information and Data

arXiv 2103.13127 ↗First posted 24 Mar 2021cs.CR cs.CV cs.LGcited by 71

Not on the map. Why?

Abstract (arXiv)

Although deep neural networks (DNNs) have made rapid progress in recent years, they are vulnerable in adversarial environments. A malicious backdoor could be embedded in a model by poisoning the training dataset, whose intention is to make the infected model give wrong predictions during inference when the specific trigger appears. To mitigate the potential threats of backdoor attacks, various backdoor detection and defense methods have been proposed. However, the existing techniques usually require the poisoned training data or access to the white-box model, which is commonly unavailable in practice. In this paper, we propose a black-box backdoor detection (B3D) method to identify backdoor attacks with only query access to the model. We introduce a gradient-free optimization algorithm to reverse-engineer the potential trigger for each class, which helps to reveal the existence of backdoor attacks. In addition to backdoor detection, we also propose a simple strategy for reliable predictions using the identified backdoored models. Extensive experiments on hundreds of DNN models trained on several datasets corroborate the effectiveness of our method under the black-box setting against various backdoor attacks.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). Builds on 36 of its 49 references that are in the map. First posted on arXiv 24 Mar 2021.

Builds on

36 of 49 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  6. 2013Intriguing properties of neural networkscited by 5,114
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  10. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  11. 2018How To Backdoor Federated Learningcited by 745
  12. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
24 more
  1. 2018Spectral Signatures in Backdoor Attackscited by 457
  2. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  3. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  4. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  5. 2019Hidden Trigger Backdoor Attackscited by 359
  6. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  7. 2019Label-Consistent Backdoor Attackscited by 285
  8. 2018Adversarial Attacks and Defences: A Surveycited by 259
  9. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
  10. 2017Neural Trojanscited by 151
  11. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  12. 2019Universal Litmus Patterns: Revealing Backdoor Attacks in CNNscited by 143
  13. 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
  14. 2020Adversarial Machine Learning -- Industry Perspectivescited by 121
  15. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  16. 2019TBT: Targeted Neural Network Attack with Bit Trojancited by 112
  17. 2019NATTACK: Learning the Distributions of Adversarial Examples for an Improved Black-Box Attack on Deep Neural Networkscited by 94
  18. 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 93
  19. 2019Defending Neural Backdoors via Generative Distribution Modelingcited by 82
  20. 2020Adversarial Distributional Training for Robust Deep Learningcited by 79
  21. 2018Backdooring Convolutional Neural Networks via Targeted Weight Perturbationscited by 66
  22. 2019NeuronInspect: Detecting Backdoors in Neural Networks via Output Explanationscited by 49
  23. 2019Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networkscited by 21
  24. 2020Scalable Backdoor Detection in Neural Networkscited by 15

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 13; 2023: 30; 2024: 13; 2025: 7; 2026: 4.

71 papers.

Sort:

By area:

  1. 2022Backdoor Defense via Decoupling the Training Processcited by 149
  2. 2022Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoningcited by 93
  3. 2023CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learningcited by 62
  4. 2021AEVA: Black-box Backdoor Detection Using Adversarial Extreme Value Analysiscited by 52
  5. 2023Detecting Backdoors During the Inference Stage Based on Corruption Robustness Consistencycited by 45
  6. 2023Black-box Backdoor Defense via Zero-shot Image Purificationcited by 44
  7. 2023Backdoor Defense via Adaptively Splitting Poisoned Datasetcited by 43
  8. 2023Shared Adversarial Unlearning: Backdoor Mitigation by Unlearning Shared Adversarial Examplescited by 40
  9. 2024Test-Time Backdoor Attacks on Multimodal Large Language Modelscited by 35
  10. 2022Post-Training Detection of Backdoor Attacks for Two-Class and Multi-Attack Scenarioscited by 29
  11. 2024Breaking the False Sense of Security in Backdoor Defense through Re-Activation Attackcited by 26
  12. 2022MOVE: Effective and Harmless Ownership Verification via Embedded External Featurescited by 20
38 more
  1. 2022PolicyCleanse: Backdoor Detection and Mitigation in Reinforcement Learningcited by 18
  2. 2023Sneaky Spikes: Uncovering Stealthy Backdoor Attacks in Spiking Neural Networks with Neuromorphic Datacited by 18
  3. 2023UMD: Unsupervised Model Detection for X2X Backdoor Attackscited by 17
  4. 2023Data and Model Poisoning Backdoor Attacks on Wireless Federated Learning, and the Defense Mechanisms: A Comprehensive Surveycited by 17
  5. 2023Defenses in Adversarial Machine Learning: A Surveycited by 14
  6. 2024Mitigating Backdoor Threats to Large Language Models: Advancement and Challengescited by 14
  7. 2023CBD: A Certified Backdoor Detector Based on Local Dominant Probabilitycited by 13
  8. 2023Detection and Defense of Unlearnable Examplescited by 13
  9. 2022Trap and Replace: Defending Backdoor Attacks by Trapping Them into an Easy-to-Replace Subnetworkcited by 12
  10. 2023BAFFLE: A Baseline of Backpropagation-Free Federated Learningcited by 12
  11. 2022Towards A Critical Evaluation of Robustness for Deep Learning Backdoor Countermeasurescited by 11
  12. 2022Selective Amnesia: On Efficient, High-Fidelity and Blind Suppression of Backdoor Effects in Trojaned Machine Learning Modelscited by 11
  13. 2021Accumulative Poisoning Attacks on Real-time Datacited by 10
  14. 2021Test-Time Detection of Backdoor Triggers for Poisoned Deep Neural Networkscited by 10
  15. 2023Single Image Backdoor Inversion via Robust Smoothed Classifierscited by 10
  16. 2021Detecting Backdoor Attacks Against Point Cloud Classifierscited by 9
  17. 2025Efficient Input-level Backdoor Defense on Text-to-Image Synthesis via Neuron Activation Variationcited by 8
  18. 2022Towards Effective and Robust Neural Trojan Defenses via Input Filteringcited by 7
  19. 2023Look, Listen, and Attack: Backdoor Attacks Against Video Action Recognitioncited by 7
  20. 2023FreeEagle: Detecting Complex Neural Trojans in Data-Free Casescited by 7
  21. 2023Watch Out! Simple Horizontal Class Backdoor Can Trivially Evade Defensecited by 7
  22. 2024BruSLeAttack: A Query-Efficient Score-Based Black-Box Sparse Adversarial Attackcited by 7
  23. 2023Differential Analysis of Triggers and Benign Features for Black-Box DNN Backdoor Detectioncited by 6
  24. 2023XGBD: Explanation-Guided Graph Backdoor Detectioncited by 6
  25. 2023Reward Certification for Policy Smoothed Reinforcement Learningcited by 6
  26. 2023Influencer Backdoor Attack on Semantic Segmentationcited by 5
  27. 2023A Proxy Attack-Free Strategy for Practically Improving the Poisoning Efficiency in Backdoor Attackscited by 5
  28. 2024Imperceptible Rhythm Backdoor Attacks: Exploring Rhythm Transformation for Embedding Undetectable Vulnerabilities on Speech Recognitioncited by 5
  29. 2025DeBackdoor: A Deductive Framework for Detecting Backdoor Attacks on Deep Models with Limited Datacited by 5
  30. 2022Backdooring Explainable Machine Learningcited by 4
  31. 2023Transpose Attack: Stealing Datasets with Bidirectional Trainingcited by 4
  32. 2023Towards Sample-specific Backdoor Attack with Clean Labels via Attribute Triggercited by 4
  33. 2024Graph Neural Backdoor: Fundamentals, Methodologies, Applications, and Future Directionscited by 4
  34. 2024Model X-ray:Detecting Backdoored Models via Decision Boundarycited by 3
  35. 2023Universal Soldier: Using Universal Adversarial Perturbations for Detecting Backdoor Attackscited by 2
  36. 2024Backdoor Defense through Self-Supervised and Generative Learningcited by 2
  37. 2025Revisiting Training-Inference Trigger Intensity in Backdoor Attackscited by 2
  38. 2025SifterNet: A Generalized and Model-Agnostic Trigger Purification Approachcited by 2

50 most cited of 71 All 71 (JSON)