mlsanityData to 14 Oct 2026

Adversarial robustness of sparse local Lipschitz predictors

arXiv 2202.13216 ↗First posted 26 Feb 2022cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

This work studies the adversarial robustness of parametric functions composed of a linear predictor and a non-linear representation map. % that satisfies certain stability condition. Our analysis relies on sparse local Lipschitzness (SLL), an extension of local Lipschitz continuity that better captures the stability and reduced effective dimensionality of predictors upon local perturbations. SLL functions preserve a certain degree of structure, given by the sparsity pattern in the representation map, and include several popular hypothesis classes, such as piece-wise linear models, Lasso and its variants, and deep feed-forward relu networks. % are sparse local Lipschitz. We provide a tighter robustness certificate on the minimal energy of an adversarial example, as well as tighter data-dependent non-uniform bounds on the robust generalization error of these predictors. We instantiate these results for the case of deep neural networks and provide numerical evidence that supports our results, shedding new insights into natural regularization strategies to increase the robustness of these models.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 92 of its 122 references that are in the map. First posted on arXiv 26 Feb 2022.

Builds on

92 of 122 references.

References · 92

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  4. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  5. 2013Intriguing properties of neural networkscited by 5,155
  6. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,998
  7. 2016Adversarial examples in the physical worldcited by 1,769
  8. 2016Understanding deep learning requires rethinking generalizationcited by 1,674
  9. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,517
  10. 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
  11. 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
  12. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
38 more
  1. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,318
  2. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  3. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,209
  4. 2016Practical Black-Box Attacks against Machine Learningcited by 903
  5. 2018Robustness May Be at Odds with Accuracycited by 880
  6. 2015The Limitations of Deep Learning in Adversarial Settingscited by 790
  7. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
  8. 2017Spectrally-normalized margin bounds for neural networkscited by 686
  9. 2015Distillation as a Defense to Adversarial Perturbations against Deep Neural Networkscited by 678
  10. 2017Exploring Generalization in Deep Learningcited by 600
  11. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  12. 2017Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methodscited by 546
  13. 2016Delving into Transferable Adversarial Examples and Black-box Attackscited by 536
  14. 2017Computing Nonvacuous Generalization Bounds for Deep (Stochastic) Neural Networks with Many More Parameters than Training Datacited by 533
  15. 2017Countering Adversarial Images using Input Transformationscited by 520
  16. 2019Fantastic Generalization Measures and Where to Find Themcited by 513
  17. 2017Sharp Minima Can Generalize For Deep Netscited by 466
  18. 2017Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Modelscited by 462
  19. 2016Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samplescited by 446
  20. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
  21. 2018Lipschitz regularity of deep neural networks: analysis and efficient estimationcited by 415
  22. 2018Defense-GAN: Protecting Classifiers Against Adversarial Attacks Using Generative Modelscited by 362
  23. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 330
  24. 2018Adversarially Robust Generalization Requires More Datacited by 318
  25. 2017A PAC-Bayesian Approach to Spectrally-Normalized Margin Bounds for Neural Networkscited by 314
  26. 2018Feature Denoising for Improving Adversarial Robustnesscited by 314
  27. 2018Stronger generalization bounds for deep nets via a compression approachcited by 307
  28. 2018Certified Defenses against Adversarial Examplescited by 306
  29. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  30. 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 293
  31. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 293
  32. 2017Size-Independent Sample Complexity of Neural Networkscited by 290
  33. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 255
  34. 2021User-friendly introduction to PAC-Bayes boundscited by 239
  35. 2019Uniform convergence may be unable to explain generalization in deep learningcited by 224
  36. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 195
  37. 2017Formal Guarantees on the Robustness of a Classifier against Adversarial Manipulationcited by 185
  38. 2018Physical Adversarial Examples for Object Detectorscited by 163

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 4; 2024: 3; 2025: 1; 2026: 1.

11 papers.

  1. 2022The Lazy Neuron Phenomenon: On Emergence of Activation Sparsity in Transformerscited by 122
  2. 2024Sparsing Law: Towards Large Language Models with Greater Activation Sparsitycited by 22
  3. 2022Instance-Dependent Generalization Bounds via Optimal Transportcited by 13
  4. 2023Sparsity-aware generalization theory for deep neural networkscited by 8
  5. 2023The Lipschitz-Variance-Margin Tradeoff for Enhanced Randomized Smoothingcited by 8
  6. 2023Understanding Noise-Augmented Training for Randomized Smoothingcited by 7
  7. 2024A direct proof of a unified law of robustness for Bregman divergence lossescited by 4
  8. 2025On the Interaction of Compressibility and Adversarial Robustnesscited by 2
  9. 2024Can overfitted deep neural networks in adversarial training generalize? -- An approximation viewpointcited by 1
  10. 2023On The Relationship Between Universal Adversarial Attacks And Sparse Representationscited by 0
  11. 2026Lipschitz bounds for integral kernelscited by 0