mlsanityData to 7 Oct 2026

Compositional Curvature Bounds for Deep Neural Networks

arXiv 2406.05119 ↗First posted 7 Jun 2024cs.LG cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

A key challenge that threatens the widespread use of neural networks in safety-critical applications is their vulnerability to adversarial attacks. In this paper, we study the second-order behavior of continuously differentiable deep neural networks, focusing on robustness against adversarial perturbations. First, we provide a theoretical analysis of robustness and attack certificates for deep classifiers by leveraging local gradients and upper bounds on the second derivative (curvature constant). Next, we introduce a novel algorithm to analytically compute provable upper bounds on the second derivative of neural networks. This algorithm leverages the compositional structure of the model to propagate the curvature bound layer-by-layer, giving rise to a scalable and modular approach. The proposed bound can serve as a differentiable regularizer to control the curvature of neural networks during training, thereby enhancing robustness. Finally, we demonstrate the efficacy of our method on classification tasks using the MNIST and CIFAR-10 datasets.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 24 of its 31 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

24 of 31 references.

References · 24

  1. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  2. 2013Intriguing properties of neural networkscited by 5,114
  3. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  4. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  5. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  6. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  7. 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
  8. 2018Robustness via curvature regularization, and vice versacited by 174
  9. 2021Globally-Robust Neural Networkscited by 106
  10. 2020Learning Control Barrier Functions from Expert Demonstrationscited by 88
  11. 2021Skew Orthogonal Convolutionscited by 67
  12. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 64
12 more
  1. 2023A Unified Algebraic Perspective on Lipschitz Neural Networkscited by 58
  2. 2023Direct Parameterization of Lipschitz-Bounded Deep Networkscited by 49
  3. 2020Second-Order Provable Defenses against Adversarial Attackscited by 39
  4. 2021Policy Smoothing for Provably Robust Reinforcement Learningcited by 39
  5. 2022Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networkscited by 39
  6. 2023Exploring and Exploiting Decision Boundary Dynamics for Adversarial Robustnesscited by 35
  7. 2021Pay attention to your loss: understanding misconceptions about 1-Lipschitz neural networkscited by 30
  8. 2021Low Curvature Activations Reduce Overfitting in Adversarial Trainingcited by 29
  9. 2022Efficiently Training Low-Curvature Neural Networkscited by 19
  10. 2023Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularizationcited by 19
  11. 2020Certifying Incremental Quadratic Constraints for Neural Networks via Convex Optimizationcited by 17
  12. 2022Improved techniques for deterministic l2 robustnesscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2026When Attribution Patching Lies: Diagnosis and a Second-Order Correctioncited by 2
  2. 2025Curvature Dynamic Black-box Attack: revisiting adversarial robustness via dynamic curvature estimationcited by 0
  3. 2026Certified and accurate computation of function space norms of deep neural networkscited by 0