mlsanityData to 7 Oct 2026

Backdoors Stuck At The Frontdoor: Multi-Agent Backdoor Attacks That Backfire

arXiv 2201.12211 ↗First posted 28 Jan 2022cs.LG cs.CR cs.MAcited by 4

On the map: LLM safety and security → Prompt injection and agent security

Abstract (arXiv)

Malicious agents in collaborative learning and outsourced data collection threaten the training of clean models. Backdoor attacks, where an attacker poisons a model during training to successfully achieve targeted misclassification, are a major concern to train-time robustness. In this paper, we investigate a multi-agent backdoor attack scenario, where multiple attackers attempt to backdoor a victim model simultaneously. A consistent backfiring phenomenon is observed across a wide range of games, where agents suffer from a low collective attack success rate. We examine different modes of backdoor attack configurations, non-cooperation / cooperation, joint distribution shifts, and game setups to return an equilibrium attack success rate at the lower bound. The results motivate the re-evaluation of backdoor defense research for practical environments.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Placed in Prompt injection and agent security, one of 12 directions in LLM safety and security. Builds on 52 of its 78 references that are in the map. First posted on arXiv 28 Jan 2022.

Builds on

52 of 78 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2016Wide Residual Networkscited by 3,057
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,250
  10. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 1,998
  11. 2017Understanding Black-box Predictions via Influence Functionscited by 1,714
  12. 2016Understanding deep learning requires rethinking generalizationcited by 1,657
38 more
  1. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  2. 2017BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chaincited by 1,052
  3. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  4. 2018How To Backdoor Federated Learningcited by 745
  5. 2018Learning to Learn without Forgetting by Maximizing Transfer and Minimizing Interferencecited by 511
  6. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  7. 2019Local Model Poisoning Attacks to Byzantine-Robust Federated Learningcited by 479
  8. 2018Spectral Signatures in Backdoor Attackscited by 457
  9. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 438
  10. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  11. 2018Detecting Backdoor Attacks on Deep Neural Networks by Activation Clusteringcited by 429
  12. 2020Backdoor Learning: A Surveycited by 412
  13. 2019Hidden Trigger Backdoor Attackscited by 359
  14. 2020Attack of the Tails: Yes, You Really Can Backdoor Federated Learningcited by 307
  15. 2019Can You Really Backdoor Federated Learning?cited by 250
  16. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  17. 2020Training independent subnetworks for robust predictioncited by 177
  18. 2020Blind Backdoors in Deep Learning Modelscited by 174
  19. 2021Mind the Style of Text! Adversarial and Backdoor Attacks Based on Text Style Transfercited by 147
  20. 2019Transferable Clean-Label Poisoning Attacks on Deep Neural Netscited by 129
  21. 2020Backdoor Attacks Against Deep Learning Systems in the Physical Worldcited by 129
  22. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  23. 2020Backdoor Attacks and Countermeasures on Deep Learning: A Comprehensive Reviewcited by 120
  24. 2019TBT: Targeted Neural Network Attack with Bit Trojancited by 112
  25. 2019Cross-Domain Transferability of Adversarial Perturbationscited by 92
  26. 2018Is feature selection secure against training data poisoning?cited by 88
  27. 2020Strong Data Augmentation Sanitizes Poisoning and Backdoor Attacks Without an Accuracy Tradeoffcited by 86
  28. 2019Superposition of many models into onecited by 64
  29. 2018Model-Reuse Attacks on Deep Learning Systemscited by 47
  30. 2019Poison as a Cure: Detecting & Neutralizing Variable-Sized Backdoor Attacks in Deep Neural Networkscited by 21
  31. 2021Subnet Replacement: Deployment-stage backdoor attack against deep neural networks in gray-box settingcited by 20
  32. 2018Universal Multi-Party Poisoning Attackscited by 17
  33. 2017A Berkeley View of Systems Challenges for AIcited by 15
  34. 2019Contamination Attacks and Mitigation in Multi-Party Machine Learningcited by 13
  35. 2017Security Risks in Deep Learning Implementationscited by 12
  36. 2020Dynamic backdoor attacks against federated learningcited by 9
  37. 2019Biometric Backdoors: A Poisoning Attack Against Unsupervised Template Updatingcited by 7
  38. 2021A Synergetic Attack against Neural Network Classifiers combining Backdoor and Adversarial Examplescited by 6

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1.

4 papers.

  1. 2022Learn2Weight: Parameter Adaptation against Similar-domain Adversarial Attackscited by 5
  2. 2022Low-Loss Subspace Compression for Clean Gains against Multi-Agent Backdoor Attackscited by 4
  3. 2022Interpolating Compressed Parameter Subspacescited by 3
  4. 2023Projected Subnetworks Scale Adaptationcited by 0