mlsanityData to 7 Oct 2026

Provable Bounds on the Hessian of Neural Networks: Derivative-Preserving Reachability Analysis

arXiv 2406.04476 ↗First posted 6 Jun 2024cs.LG cs.SY eess.SYcited by 2

Not on the map. Why?

Abstract (arXiv)

We propose a novel reachability analysis method tailored for neural networks with differentiable activations. Our idea hinges on a sound abstraction of the neural network map based on first-order Taylor expansion and bounding the remainder. To this end, we propose a method to compute analytical bounds on the network's first derivative (gradient) and second derivative (Hessian). A key aspect of our method is loop transformation on the activation functions to exploit their monotonicity effectively. The resulting end-to-end abstraction locally preserves the derivative information, yielding accurate bounds on small input sets. Finally, we employ a branch and bound framework for larger input sets to refine the abstraction recursively. We evaluate our method numerically via different examples and compare the results with relevant state-of-the-art methods.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 49 references that are in the map. First posted on arXiv 6 Jun 2024.

Builds on

33 of 49 references.

References · 33

  1. 2013Intriguing properties of neural networkscited by 5,114
  2. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  3. 2016CVXPY: A Python-Embedded Modeling Language for Convex Optimizationcited by 633
  4. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  5. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  6. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  7. 2019Efficient and Accurate Estimation of Lipschitz Constants for Deep Neural Networkscited by 290
  8. 2018Towards Fast Computation of Certified Robustness for ReLU Networkscited by 254
  9. 2021Beta-CROWN: Efficient Bound Propagation with Per-neuron Split Constraints for Complete and Incomplete Neural Network Robustness Verificationcited by 224
  10. 2018Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approachcited by 181
  11. 2020Exactly Computing the Local Lipschitz Constant of ReLU Networkscited by 102
  12. 2019Safety Verification and Robustness Analysis of Neural Networks via Quadratic Constraints and Semidefinite Programmingcited by 100
21 more
  1. 2020Lipschitz constant estimation of Neural Networks via sparse polynomial optimizationcited by 87
  2. 2018Verisig: verifying safety properties of hybrid systems with neural network controllerscited by 83
  3. 2022Efficiently Computing Local Lipschitz Constants of Neural Networks via Bound Propagationcited by 45
  4. 2020Reach-SDP: Reachability Analysis of Closed-Loop Systems with Neural Network Controllers via Semidefinite Programmingcited by 43
  5. 2020Second-Order Provable Defenses against Adversarial Attackscited by 39
  6. 2021OVERT: An Algorithm for Safety Verification of Neural Network Control Policies for Nonlinear Systemscited by 37
  7. 2020Reachable Set Estimation for Neural Network Control Systems: A Simulation-Guided Approachcited by 36
  8. 2020Reachable Polyhedral Marching (RPM): A Safety Verification Algorithm for Robotic Systems with Deep Neural Network Componentscited by 36
  9. 2020Lagrangian Decomposition for Neural Network Verificationcited by 35
  10. 2018RecurJac: An Efficient Recursive Algorithm for Bounding Jacobian Matrix of Neural Networks and Its Applicationscited by 27
  11. 2023Certified Robustness via Dynamic Margin Maximization and Improved Lipschitz Regularizationcited by 19
  12. 2017General Heuristics for Nonconvex Quadratically Constrained Quadratic Programmingcited by 17
  13. 2019Fast Neural Network Verification via Shadow Pricescited by 17
  14. 2020Certifying Incremental Quadratic Constraints for Neural Networks via Convex Optimizationcited by 17
  15. 2020Tightened Convex Relaxations for Neural Network Robustness Certificationcited by 14
  16. 2021Efficient Reachability Analysis of Closed-Loop Systems with Neural Network Controllerscited by 10
  17. 2021DeepSplit: Scalable Verification of Deep Neural Networks via Operator Splittingcited by 8
  18. 2022ReachLipBnB: A branch-and-bound method for reachability analysis of neural autonomous systems using Lipschitz boundscited by 6
  19. 2023Contraction-Guided Adaptive Partitioning for Reachability Analysis of Neural Network Controlled Systemscited by 3
  20. 2022Automated Reachability Analysis of Neural Network-Controlled Systems via Adaptive Polytopescited by 2
  21. 2023On the Difficulty of Intersection Checking with Polynomial Zonotopescited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Certified and accurate computation of function space norms of deep neural networkscited by 0
  2. 2026High Probability Derivative Bounds for Random tanh Neural Networks on a Hypercubecited by 0