mlsanityData to 14 Oct 2026

SEA: Shareable and Explainable Attribution for Query-based Black-box Attacks

arXiv 2308.11845 ↗First posted 23 Aug 2023cs.LG cs.CRcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine Learning (ML) systems are vulnerable to adversarial examples, particularly those from query-based black-box attacks. Despite various efforts to detect and prevent such attacks, ML systems are still at risk, demanding a more comprehensive approach to security that includes logging, analyzing, and sharing evidence. While traditional security benefits from well-established practices of forensics and threat intelligence sharing, ML security has yet to find a way to profile its attackers and share information about them. In response, this paper introduces SEA, a novel ML security system to characterize black-box attacks on ML systems for forensic purposes and to facilitate human-explainable intelligence sharing. SEA leverages Hidden Markov Models to attribute the observed query sequence to known attacks. It thus understands the attack's progression rather than focusing solely on the final adversarial examples. Our evaluations reveal that SEA is effective at attack attribution, even on the second incident, and is robust to adaptive strategies designed to evade forensic analysis. SEA's explanations of the attack's behavior allow us even to fingerprint specific minor bugs in widely used attack libraries. For example, we discover that the SignOPT and Square attacks in ART v1.14 send over 50% duplicated queries. We thoroughly evaluate SEA on a variety of settings and demonstrate that it can recognize the same attack with more than 90% Top-1 and 95% Top-3 accuracy. Finally, we demonstrate how SEA generalizes to other domains like text classification.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 56 of its 86 references that are in the map. First posted on arXiv 23 Aug 2023.

Builds on

56 of 86 references.

References · 56

  1. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2014Deep Learning Face Attributes in the Wildcited by 3,693
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  8. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  9. 2017Evasion Attacks against Machine Learning at Test Timecited by 689
  10. 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 685
  11. 2019Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailmentcited by 671
  12. 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 666
38 more
  1. 2020RobustBench: a standardized adversarial robustness benchmarkcited by 661
  2. 2016Stealing Machine Learning Models via Prediction APIscited by 619
  3. 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 599
  4. 2020On Adaptive Attacks to Adversarial Example Defensescited by 555
  5. 2017Countering Adversarial Images using Input Transformationscited by 520
  6. 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 501
  7. 2022Diffusion Models for Adversarial Purificationcited by 476
  8. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  9. 2019Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attackscited by 442
  10. 2017Mitigating Adversarial Effects Through Randomizationcited by 383
  11. 2018TextBugger: Generating Adversarial Text Against Real-world Applicationscited by 381
  12. 2018Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifierscited by 345
  13. 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 333
  14. 2019Simple Black-box Adversarial Attackscited by 251
  15. 2018Adversarial Robustness Toolbox v1.0.0cited by 215
  16. 2023Better Diffusion Models Further Improve Adversarial Trainingcited by 208
  17. 2019Certified Robustness to Adversarial Word Substitutionscited by 168
  18. 2020Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNetscited by 167
  19. 2018PRADA: Protecting against DNN Model Stealing Attackscited by 165
  20. 2018Query-Efficient Hard-label Black-box Attack:An Optimization-based Approachcited by 130
  21. 2019Sign-OPT: A Query-Efficient Hard-label Adversarial Attackcited by 120
  22. 2020QEBA: Query-Efficient Boundary-Based Blackbox Attackcited by 104
  23. 2023Frontier AI Regulation: Managing Emerging Risks to Public Safetycited by 98
  24. 2020RayS: A Ray Searching Method for Hard-label Adversarial Attackcited by 96
  25. 2020GeoDA: a geometric framework for black-box adversarial attackscited by 66
  26. 2020An Evaluation of Change Point Detection Algorithmscited by 63
  27. 2019Stateful Detection of Black-Box Adversarial Attackscited by 56
  28. 2019Parsimonious Black-Box Adversarial Attacks via Efficient Combinatorial Optimizationcited by 49
  29. 2021Random Noise Defense Against Query-Based Black-Box Attackscited by 47
  30. 2021Poison Forensics: Traceback of Data Poisoning Attacks in Neural Networkscited by 40
  31. 2020Blacklight: Scalable Defense for Neural Networks against Query-Based Black-Box Attackscited by 37
  32. 2022On the Limitations of Stochastic Pre-processing Defensescited by 21
  33. 2020Projection & Probability-Driven Black-Box Attackcited by 19
  34. 2022Adversarial Attack on Attackers: Post-Process to Mitigate Black-Box Score-Based Query Attackscited by 19
  35. 2021On the Effectiveness of Small Input Noise for Defending Against Query-based Black-Box Attackscited by 17
  36. 2022Demystifying the Adversarial Robustness of Random Transformation Defensescited by 14
  37. 2023Evading Black-box Classifiers Without Breaking Eggscited by 13
  38. 2020AdvMind: Inferring Adversary Intent of Black-Box Attackscited by 12

50 most cited of 56 All 56 (JSON)

Built on by

Not cited yet.