mlsanityData to 7 Oct 2026

Backdooring Explainable Machine Learning

arXiv 2204.09498 ↗First posted 20 Apr 2022cs.CR cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Explainable machine learning holds great potential for analyzing and understanding learning-based systems. These methods can, however, be manipulated to present unfaithful explanations, giving rise to powerful and stealthy adversaries. In this paper, we demonstrate blinding attacks that can fully disguise an ongoing attack against the machine learning model. Similar to neural backdoors, we modify the model's prediction upon trigger presence but simultaneously also fool the provided explanation. This enables an adversary to hide the presence of the trigger or point the explanation to entirely different portions of the input, throwing a red herring. We analyze different manifestations of such attacks for different explanation types in the image domain, before we resume to conduct a red-herring attack against malware classification.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 67 of its 106 references that are in the map. First posted on arXiv 20 Apr 2022.

Builds on

67 of 106 references.

References · 67

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  5. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  8. 2013Intriguing properties of neural networkscited by 5,114
  9. 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
  10. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  11. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  12. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
38 more
  1. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  2. 2017Improved Training of Wasserstein GANscited by 2,223
  3. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  4. 2017Learning Important Features Through Propagating Activation Differencescited by 1,454
  5. 2018Sanity Checks for Saliency Mapscited by 1,149
  6. 2017Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoningcited by 1,012
  7. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
  8. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  9. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  10. 2017Ensemble Adversarial Training: Attacks and Defensescited by 833
  11. 2015The Limitations of Deep Learning in Adversarial Settingscited by 784
  12. 2016Universal adversarial perturbationscited by 774
  13. 2018How To Backdoor Federated Learningcited by 745
  14. 2019Adversarial Training for Free!cited by 638
  15. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  16. 2019Score-CAM: Score-Weighted Visual Explanations for Convolutional Neural Networkscited by 575
  17. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  18. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 542
  19. 2017Adversarial Patchcited by 514
  20. 2018Poison Frogs! Targeted Clean-Label Poisoning Attacks on Neural Networkscited by 500
  21. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
  22. 2017Interpretation of Neural Networks is Fragilecited by 443
  23. 2019Fooling LIME and SHAP: Adversarial Attacks on Post hoc Explanation Methodscited by 440
  24. 2019STRIP: A Defence Against Trojan Attacks on Deep Neural Networkscited by 437
  25. 2017Wild Patterns: Ten Years After the Rise of Adversarial Machine Learningcited by 411
  26. 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 359
  27. 2020Invisible Backdoor Attack with Sample-Specific Triggerscited by 338
  28. 2017The (Un)reliability of saliency methodscited by 331
  29. 2020Input-Aware Dynamic Backdoor Attackcited by 304
  30. 2016Not Just a Black Box: Learning Important Features Through Propagating Activation Differencescited by 295
  31. 2018Manipulating Machine Learning: Poisoning Attacks and Countermeasures for Regression Learningcited by 219
  32. 2019Explanations can be manipulated and geometry is to blamecited by 215
  33. 2017The Shattered Gradients Problem: If resnets are the answer, then what is the question?cited by 156
  34. 2021Rethinking the Backdoor Attacks' Triggers: A Frequency Perspectivecited by 155
  35. 2018SentiNet: Detecting Localized Universal Attacks Against Deep Learning Systemscited by 148
  36. 2020Dynamic Backdoor Attacks Against Machine Learning Modelscited by 147
  37. 2021BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learningcited by 120
  38. 2019Fooling Neural Network Interpretations via Adversarial Model Manipulationcited by 112

50 most cited of 67 All 67 (JSON)

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 1; 2024: 1.

4 papers.

  1. 2022Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesiscited by 59
  2. 2023Don't trust your eyes: on the (un)reliability of feature visualizationscited by 32
  3. 2022Unfooling Perturbation-Based Post Hoc Explainerscited by 12
  4. 2024Attention Masks Help Adversarial Attacks to Bypass Safety Detectorscited by 0