mlsanityData to 14 Oct 2026

Revisiting the Relation Between Robustness and Universality

arXiv 2510.19427 ↗First posted 22 Oct 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

The modified universality hypothesis proposed by Jones et al. (2022) suggests that adversarially robust models trained for a given task are highly similar. We revisit the hypothesis and test its generality. While we verify Jones' main claim of high representational similarity in specific settings, results are not consistent across different datasets. We also discover that predictive behavior does not converge with increasing robustness and thus is not universal. We find that differing predictions originate in the classification layer, but show that more universal predictive behavior can be achieved with simple retraining of the classifiers. Overall, our work points towards partial universality of neural networks in specific settings and away from notions of strict universality.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 40 of its 56 references that are in the map. First posted on arXiv 22 Oct 2025.

Builds on

40 of 56 references.

References · 40

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  3. 2016Densely Connected Convolutional Networkscited by 6,720
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2016Wide Residual Networkscited by 3,070
  7. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,546
  8. 2019Similarity of Neural Network Representations Revisitedcited by 1,755
  9. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 953
  10. 2019Deep Ensembles: A Loss Landscape Perspectivecited by 443
  11. 2020Do Adversarially Robust ImageNet Models Transfer Better?cited by 316
  12. 2015Convergent Learning: Do different neural networks learn the same representations?cited by 220
28 more
  1. 2021Revisiting Model Stitching to Compare Neural Representationscited by 209
  2. 2022Relative representations enable zero-shot latent space communicationcited by 191
  3. 2019Predictive Multiplicity in Classificationcited by 166
  4. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 156
  5. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 144
  6. 2023Getting aligned on representational alignmentcited by 143
  7. 2023Similarity of Neural Network Models: A Survey of Functional and Representational Measurescited by 131
  8. 2021Generalized Shape Metrics on Neural Representationscited by 112
  9. 2023Circuit Component Reuse Across Tasks in Transformer Language Modelscited by 108
  10. 2024Universal Neurons in GPT2 Language Modelscited by 102
  11. 2019BERTs of a feather do not generalize together: Large variability in generalization across models with similar test set performancecited by 89
  12. 2022Reliability of CKA as a Similarity Measure in Deep Learningcited by 87
  13. 2021Similarity and Matching of Neural Network Representationscited by 78
  14. 2023Successor Heads: Recurring, Interpretable Attention Heads In The Wildcited by 77
  15. 2021Representation Topology Divergence: A Method for Comparing Neural Network Representationscited by 66
  16. 2022Can Neural Nets Learn the Same Model Twice? Investigating Reproducibility and Double Descent from the Decision Boundary Perspectivecited by 57
  17. 2022Mechanistic Mode Connectivitycited by 50
  18. 2021Nondeterminism and Instability in Neural Network Optimizationcited by 48
  19. 2021Leave-one-out Unfairnesscited by 36
  20. 2022Deconfounded Representation Similarity for Comparison of Neural Networkscited by 19
  21. 2024ReSi: A Comprehensive Benchmark for Representational Similarity Measurescited by 19
  22. 2023Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Modelscited by 18
  23. 2023Understanding the Inner Workings of Language Models Through Representation Dissimilaritycited by 12
  24. 2024Evaluating Representational Similarity Measures from the Lens of Functional Correspondencecited by 12
  25. 2022On the Origins of the Block Structure Phenomenon in Neural Network Representationscited by 10
  26. 2022Model Stability with Continuous Data Updatescited by 9
  27. 2022On the Prediction Instability of Graph Neural Networkscited by 6
  28. 2022Disentangling Model Multiplicity in Deep Learningcited by 5

Built on by

Not cited yet.