mlsanityData to 7 Oct 2026

BackdoorBench: A Comprehensive Benchmark and Analysis of Backdoor Learning

arXiv 2407.19845 ↗First posted 29 Jul 2024cs.LG cs.CRcited by 17

Not on the map. Why?

Abstract (arXiv)

As an emerging approach to explore the vulnerability of deep neural networks (DNNs), backdoor learning has attracted increasing interest in recent years, and many seminal backdoor attack and defense algorithms are being developed successively or concurrently, in the status of a rapid arms race. However, mainly due to the diverse settings, and the difficulties of implementation and reproducibility of existing works, there is a lack of a unified and standardized benchmark of backdoor learning, causing unfair comparisons or unreliable conclusions (e.g., misleading, biased or even false conclusions). Consequently, it is difficult to evaluate the current progress and design the future development roadmap of this literature. To alleviate this dilemma, we build a comprehensive benchmark of backdoor learning called BackdoorBench. Our benchmark makes three valuable contributions to the research community. 1) We provide an integrated implementation of state-of-the-art (SOTA) backdoor learning algorithms (currently including 20 attack and 32 defense algorithms), based on an extensible modular-based codebase. 2) We conduct comprehensive evaluations with 5 poisoning ratios, based on 4 models and 4 datasets, leading to 11,492 pairs of attack-against-defense evaluations in total. 3) Based on above evaluations, we present abundant analysis from 10 perspectives via 18 useful analysis tools, and provide several inspiring insights about backdoor learning. We hope that our efforts could build a solid foundation of backdoor learning to facilitate researchers to investigate existing algorithms, develop more innovative algorithms, and explore the intrinsic mechanism of backdoor learning. Finally, we have created a user-friendly website at http://backdoorbench.com, which collects all important information of BackdoorBench, including codebase, docs, leaderboard, and model Zoo.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Builds on 90 of its 129 references that are in the map. First posted on arXiv 29 Jul 2024.

Builds on

90 of 129 references.

References · 90

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2022A ConvNet for the 2020scited by 3,545
  8. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  9. 2016Adversarial examples in the physical worldcited by 1,762
  10. 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
  11. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
  12. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
38 more
  1. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  2. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 974
  3. 2018How To Backdoor Federated Learningcited by 745
  4. 2018An Empirical Study of Example Forgetting during Deep Neural Network Learningcited by 669
  5. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 659
  6. 2017Exploring Generalization in Deep Learningcited by 595
  7. 2018Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networkscited by 550
  8. 2018Spectral Signatures in Backdoor Attackscited by 457
  9. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  10. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  11. 2018Analyzing Federated Learning through an Adversarial Lenscited by 346
  12. 2020Reflection Backdoor: A Natural Backdoor Attack on Deep Neural Networkscited by 341
  13. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  14. 2021Neural Attention Distillation: Erasing Backdoor Triggers from Deep Neural Networkscited by 313
  15. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
  16. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  17. 2019Label-Consistent Backdoor Attackscited by 285
  18. 2021Anti-Backdoor Learning: Training Clean Models on Poisoned Datacited by 258
  19. 2019A new Backdoor Attack in CNNs by training set corruption without label poisoningcited by 237
  20. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 230
  21. 2018Adversarial Robustness Toolbox v1.0.0cited by 214
  22. 2020Blind Backdoors in Deep Learning Modelscited by 174
  23. 2020Clean-Label Backdoor Attacks on Video Recognition Modelscited by 162
  24. 2021Adversarial Unlearning of Backdoors via Implicit Hypergradientcited by 160
  25. 2021Rethinking the Backdoor Attacks' Triggers: A Frequency Perspectivecited by 155
  26. 2017Neural Trojanscited by 151
  27. 2022Backdoor Defense via Decoupling the Training Processcited by 149
  28. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  29. 2020Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustnesscited by 130
  30. 2022BackdoorBench: A Comprehensive Benchmark of Backdoor Learningcited by 124
  31. 2021SPECTRE: Defending Against Backdoor Attacks Using Robust Statisticscited by 123
  32. 2019Demon in the Variant: Statistical Analysis of DNNs for Robust Backdoor Contamination Detectioncited by 121
  33. 2020Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Reviewcited by 120
  34. 2021Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratchcited by 98
  35. 2022Neurotoxin: Durable Backdoors in Federated Learningcited by 97
  36. 2019Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacycited by 93
  37. 2023SCALE-UP: An Efficient Black-box Input-level Backdoor Detection via Analyzing Scaled Prediction Consistencycited by 93
  38. 2022BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learningcited by 82

50 most cited of 90 All 90 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 6; 2026: 8.

17 papers.

Sort:

By area:

  1. 2026SkillTrojan: Backdoor Attacks on Skill-Based Agent Systemscited by 21
  2. 2024Mitigating the Backdoor Effect for Multi-Task Model Merging via Safety-Aware Subspacecited by 14
  3. 2025BadVideo: Stealthy Backdoor Attack against Text-to-Video Generationcited by 9
  4. 2025CLIP-Guided Backdoor Defense through Entropy-Based Poisoned Dataset Separationcited by 9
  5. 2025Breaking the Stealth-Potency Trade-off in Clean-Image Backdoors with Generative Trigger Optimizationcited by 7
  6. 2026From Internal Diagnosis to External Auditing: A VLM-Driven Paradigm for Data-Free Online Backdoor Defensecited by 3
  7. 2024Reliable Poisoned Sample Detection against Backdoor Attacks Enhanced by Sharpness Aware Minimizationcited by 2
  8. 2025SifterNet: A Generalized and Model-Agnostic Trigger Purification Approachcited by 2
  9. 2025Class-Conditional Neural Polarizer: A Lightweight and Effective Backdoor Defense by Purifying Poisoned Featurescited by 1
  10. 2025DualEdit: Mitigating Safety Fallback in LLM Backdoor Editing via Affirmation-Refusal Regulationcited by 1
  11. 2026Backdoor Directions in Vision Transformerscited by 1
  12. 2026From Parameters to Feature Space: Task Arithmetic for Backdoor Mitigation in Model Mergingcited by 1
5 more
  1. 2024Trading Devil RL: Backdoor attack via Stock market, Bayesian Optimization and Reinforcement Learningcited by 0
  2. 2026TCAP: Tri-Component Attention Profiling for Unsupervised Backdoor Detection in MLLM Fine-Tuningcited by 0
  3. 2026Kill it with FIRE: On Leveraging Latent Space Directions for Runtime Backdoor Mitigation in Deep Neural Networkscited by 0
  4. 2026TimeGuard: Channel-wise Pool Training for Backdoor Defense in Time Series Forecastingcited by 0
  5. 2026Robust Watermarks Meet Backdoored Models: Evading Diffusion Semantic Watermarks via Stealthy Backdoorcited by 0