mlsanityData to 7 Oct 2026

Similarity of Neural Architectures using Adversarial Attack Transferability

arXiv 2210.11407 ↗First posted 20 Oct 2022cs.LG cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

In recent years, many deep neural architectures have been developed for image classification. Whether they are similar or dissimilar and what factors contribute to their (dis)similarities remains curious. To address this question, we aim to design a quantitative and scalable similarity measure between neural architectures. We propose Similarity by Attack Transferability (SAT) from the observation that adversarial attack transferability contains information related to input gradients and decision boundaries widely used to understand model behaviors. We conduct a large-scale analysis on 69 state-of-the-art ImageNet classifiers using our proposed similarity function to answer the question. Moreover, we observe neural architecture-related phenomena using model similarity that model diversity can lead to better performance on model ensembles and knowledge distillation under specific conditions. Our results provide insights into why developing diverse neural architectures with distinct components is necessary.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 114 of its 137 references that are in the map. First posted on arXiv 20 Oct 2022.

Builds on

114 of 137 references.

References · 114

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  7. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  8. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  9. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  10. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  11. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  12. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
38 more
  1. 2016Layer Normalizationcited by 5,579
  2. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  3. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  7. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  8. 2022A ConvNet for the 2020scited by 3,545
  9. 2017Axiomatic Attribution for Deep Networkscited by 3,350
  10. 2016Wide Residual Networkscited by 3,057
  11. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  12. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  13. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  14. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  15. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  16. 2019Similarity of Neural Network Representations Revisitedcited by 1,695
  17. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
  18. 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
  19. 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
  20. 2018Group Normalizationcited by 1,514
  21. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  22. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  23. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,417
  24. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  25. 2017Visualizing the Loss Landscape of Neural Netscited by 1,126
  26. 2017SmoothGrad: removing noise by adding noisecited by 1,097
  27. 2020In Search of Lost Domain Generalizationcited by 1,042
  28. 2014Striving for Simplicity: The All Convolutional Netcited by 983
  29. 2020Designing Network Design Spacescited by 954
  30. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  31. 2018Robustness May Be at Odds with Accuracycited by 873
  32. 2021CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classificationcited by 806
  33. 2021Do Vision Transformers See Like Convolutional Neural Networks?cited by 671
  34. 2021Transformer in Transformercited by 632
  35. 2019ECA-Net: Efficient Channel Attention for Deep Convolutional Neural Networkscited by 572
  36. 2019Improved Knowledge Distillation via Teacher Assistantcited by 567
  37. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566
  38. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565

50 most cited of 114 All 114 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 2; 2025: 2.

4 papers.

  1. 2024RobustBlack: Challenging Black-Box Adversarial Attacks on State-of-the-Art Defensescited by 2
  2. 2025Are Time-Series Foundation Models Deployment-Ready? A Systematic Study of Adversarial Robustness Across Domainscited by 2
  3. 2025SC-Pro: Training-Free Framework for Defending Unsafe Image Synthesis Attackcited by 1
  4. 2024Bias Similarity Measurement: A Black-Box Audit of Fairness Across LLMscited by 0