mlsanityData to 7 Oct 2026

Diagnostics for Deep Neural Networks with Automated Copy/Paste Attacks

arXiv 2211.10024 ↗First posted 18 Nov 2022cs.LG cs.AI cs.CRcited by 6

Not on the map. Why?

Abstract (arXiv)

This paper considers the problem of helping humans exercise scalable oversight over deep neural networks (DNNs). Adversarial examples can be useful by helping to reveal weaknesses in DNNs, but they can be difficult to interpret or draw actionable conclusions from. Some previous works have proposed using human-interpretable adversarial attacks including copy/paste attacks in which one natural image pasted into another causes an unexpected misclassification. We build on these with two contributions. First, we introduce Search for Natural Adversarial Features Using Embeddings (SNAFUE) which offers a fully automated method for finding copy/paste attacks. Second, we use SNAFUE to red team an ImageNet classifier. We reproduce copy/paste attacks from previous works and find hundreds of other easily-describable vulnerabilities, all without a human in the loop. Code is available at https://github.com/thestephencasper/snafue

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 38 of its 73 references that are in the map. First posted on arXiv 18 Nov 2022.

Builds on

38 of 73 references.

References · 38

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  6. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  7. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  8. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  9. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  10. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  11. 2019Natural Adversarial Examplescited by 1,401
  12. 2017Towards A Rigorous Science of Interpretable Machine Learningcited by 1,400
26 more
  1. 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
  2. 2022Red Teaming Language Models with Language Modelscited by 983
  3. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  4. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  5. 2021Knowledge Neurons in Pretrained Transformerscited by 590
  6. 2017Adversarial Patchcited by 514
  7. 2019Risks from Learned Optimization in Advanced Machine Learning Systemscited by 250
  8. 2020Compositional Explanations of Neuronscited by 172
  9. 2022Domino: Discovering Systematic Errors with Cross-Modal Embeddingscited by 155
  10. 2022Natural Language Descriptions of Deep Visual Featurescited by 136
  11. 2018Constructing Unrestricted Adversarial Examples with Generative Modelscited by 135
  12. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  13. 2019TABOR: A Highly Accurate Approach to Inspecting and Restoring Trojan Backdoors in AI Systemscited by 120
  14. 2020Neuron Shapley: Discovering the Responsible Neuronscited by 98
  15. 2021A Survey on Bias in Visual Datasetscited by 84
  16. 2021Leveraging Sparse Linear Layers for Debuggable Deep Networkscited by 82
  17. 2022Distilling Model Failures as Directions in Latent Spacecited by 82
  18. 2019Explanation by Progressive Exaggerationcited by 76
  19. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  20. 2019Semantic Adversarial Attacks: Parametric Transformations That Fool Deep Classifierscited by 56
  21. 20213DB: A Framework for Debugging Computer Vision Modelscited by 31
  22. 2017Towards Interpretable Deep Neural Networks by Leveraging Adversarial Examplescited by 27
  23. 2021Robust Feature-Level Adversaries are Interpretability Toolscited by 27
  24. 2018xGEMs: Generating Examplars to Explain Black-Box Modelscited by 21
  25. 2020An overview of 11 proposals for building safe advanced AIcited by 18
  26. 2020Generating Semantic Adversarial Examples via Feature Manipulationcited by 4

Built on by

Cited by year

Citing papers per year: 2024: 6.

6 papers.

  1. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 415
  2. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
  3. 2024Red-Teaming for Generative AI: Silver Bullet or Security Theater?cited by 81
  4. 2024A Multimodal Automated Interpretability Agentcited by 62
  5. 2024The SaTML '24 CNN Interpretability Competition: New Innovations for Concept-Level Interpretabilitycited by 3
  6. 2024Exploring higher-order neural network node interactions with total correlationcited by 0