Model Mimic Attack: Knowledge Distillation for Provably Transferable Adversarial Examples
Not on the map. Why?
Abstract (arXiv)
The vulnerability of artificial neural networks to adversarial perturbations in the black-box setting is widely studied in the literature. The majority of attack methods to construct these perturbations suffer from an impractically large number of queries required to find an adversarial example. In this work, we focus on knowledge distillation as an approach to conduct transfer-based black-box adversarial attacks and propose an iterative training of the surrogate model on an expanding dataset. This work is the first, to our knowledge, to provide provable guarantees on the success of knowledge distillation-based attack on classification neural networks: we prove that if the student model has enough learning capabilities, the attack on the teacher model is guaranteed to be found within the finite number of distillation iterations.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 37 of its 45 references that are in the map. First posted on arXiv 21 Oct 2024.
Builds on
37 of 45 references.
References · 37
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2015Distilling the Knowledge in a Neural Networkcited by 10,076
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
- 2013Intriguing properties of neural networkscited by 5,114
- 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2019Square Attack: a query-efficient black-box adversarial attack via random searchcited by 682
- 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 675
- 2017ZOO: Zeroth Order Optimization based Black-box Attacks to Deep Neural Networks without Training Substitute Modelscited by 656
- 2018Improving Transferability of Adversarial Examples with Input Diversitycited by 591
- 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 530
25 more
- 2019Patient Knowledge Distillation for BERT Model Compressioncited by 502
- 2018Black-box Adversarial Attacks with Limited Queries and Informationcited by 499
- 2019HopSkipJumpAttack: A Query-Efficient Decision-Based Attackcited by 328
- 2019Simple Black-box Adversarial Attackscited by 249
- 2020Skip Connections Matter: On the Transferability of Adversarial Examples Generated with ResNetscited by 165
- 2018Prior Convictions: Black-Box Adversarial Attacks with Bandits and Priorscited by 145
- 2019Adversarially Robust Distillationcited by 124
- 2019Deep Neural Network Fingerprinting by Conferrable Adversarial Examplescited by 90
- 2021Revisiting Adversarial Robustness Distillation: Robust Soft Labels Make Student Bettercited by 77
- 2023Knowledge-Augmented Reasoning Distillation for Small Language Models in Knowledge-Intensive Taskscited by 62
- 2023Knowledge Diffusion for Distillationcited by 51
- 2022LGV: Boosting Adversarial Example Transferability from Large Geometric Vicinitycited by 39
- 2018Private Model Compression via Knowledge Distillationcited by 37
- 2020Improving Query Efficiency of Black-box Adversarial Attackcited by 34
- 2023Making Substitute Models More Bayesian Can Enhance Transferability of Adversarial Examplescited by 32
- 2023Generalizable Black-Box Adversarial Attack with Meta Learningcited by 29
- 2020ES Attack: Model Stealing against Deep Neural Networks without Data Hurdlescited by 24
- 2023Enhancing Generalization of Universal Adversarial Perturbation through Gradient Aggregationcited by 22
- 2023BlackboxBench: A Comprehensive Benchmark of Black-box Adversarial Attackscited by 21
- 2021Training Meta-Surrogate Model for Transferable Adversarial Attackcited by 15
- 2023Improving the Transferability of Adversarial Examples via Direction Tuningcited by 9
- 2024Probabilistically Robust Watermarking of Neural Networkscited by 7
- 2024BruSLeAttack: A Query-Efficient Score-Based Black-Box Sparse Adversarial Attackcited by 7
- 2021Knowledge Cross-Distillation for Membership Privacycited by 6
- 2024GLiRA: Black-Box Membership Inference Attack via Knowledge Distillationcited by 6
Built on by
Citing papers per year: 2025: 2.
2 papers.