mlsanityData to 14 Oct 2026

Expressive Losses for Verified Robustness via Convex Combinations

arXiv 2305.13991 ↗First posted 23 May 2023cs.LG cs.CR stat.MLcited by 23

Not on the map. Why?

Abstract (arXiv)

In order to train networks for verified adversarial robustness, it is common to over-approximate the worst-case loss over perturbation regions, resulting in networks that attain verifiability at the expense of standard performance. As shown in recent work, better trade-offs between accuracy and robustness can be obtained by carefully coupling adversarial training with over-approximations. We hypothesize that the expressivity of a loss function, which we formalize as the ability to span a range of trade-offs between lower and upper bounds to the worst-case loss through a single parameter (the over-approximation coefficient), is key to attaining state-of-the-art performance. To support our hypothesis, we show that trivial expressive losses, obtained via convex combinations between adversarial attacks and IBP bounds, yield state-of-the-art results across a variety of settings in spite of their conceptual simplicity. We provide a detailed analysis of the relationship between the over-approximation coefficient and performance profiles across different expressive losses, showing that, while expressivity is essential, better approximations of the worst-case loss are not necessarily linked to superior robustness-accuracy trade-offs.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Builds on 58 of its 85 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

58 of 85 references.

References · 58

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  5. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 7,005
  6. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  7. 2013Intriguing properties of neural networkscited by 5,155
  8. 2021Convex optimizationcited by 2,515
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,451
  10. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,340
  11. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,268
  12. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
38 more
  1. 2020Fast is better than free: Revisiting adversarial trainingcited by 727
  2. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 567
  3. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
  4. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 377
  5. 2019Provably Robust Deep Learning via Adversarially Trained Smoothed Classifierscited by 330
  6. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
  7. 2017Formal Verification of Piece-Wise Linear Feed-Forward Neural Networkscited by 242
  8. 2018Adversarial Risk and the Dangers of Evaluating Against Weak Attackscited by 221
  9. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 205
  10. 2020Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifierscited by 187
  11. 2018Semidefinite relaxations for certifying robustness to adversarial examplescited by 179
  12. 2018Scaling provable adversarial defensescited by 143
  13. 2019Branch and Bound for Piecewise Linear Neural Network Verificationcited by 138
  14. 2017An approach to reachability analysis for feed-forward ReLU neural networkscited by 127
  15. 2018Strong mixed-integer programming formulations for trained neural networkscited by 115
  16. 2021Globally-Robust Neural Networkscited by 108
  17. 2021Orthogonalizing Convolutional Layers with the Cayley Transformcited by 105
  18. 2022General Cutting Planes for Bound-Propagation-Based Neural Network Verificationcited by 103
  19. 2022Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Boundcited by 92
  20. 2023Revisiting Adversarial Training for ImageNet: Architectures, Training and Generalization across Threat Modelscited by 79
  21. 2021PRIMA: General and Precise Neural Network Certification via Scalable Convex Hull Approximationscited by 77
  22. 2021The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Resultscited by 71
  23. 2018Training for Faster Adversarial Robustness Verification via Inducing ReLU Stabilitycited by 67
  24. 2021Skew Orthogonal Convolutionscited by 67
  25. 2022In Defense of the Unitary Scalarization for Deep Multi-Task Learningcited by 67
  26. 2020Enabling certification of verification-agnostic networks via memory-efficient semidefinite programmingcited by 66
  27. 2021Training Certifiably Robust Neural Networks with Efficient Local Lipschitz Boundscited by 65
  28. 2022Do Current Multi-Task Optimization Methods in Deep Learning Even Help?cited by 60
  29. 2022Certified Training: Small Boxes are All You Needcited by 52
  30. 2022Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspectivecited by 51
  31. 2020The Convex Relaxation Barrier, Revisited: Tightened Single-Neuron Relaxations for Neural Network Verificationcited by 50
  32. 2021Improved deterministic l2 robustness on CIFAR-10 and CIFAR-100cited by 47
  33. 2021Fast Certified Robust Training with Short Warmupcited by 46
  34. 2021Improved Branch and Bound for Neural Network Verification via Lagrangian Decompositioncited by 44
  35. 2021Towards Certifying L-infinity Robustness using Neural Networks with L-inf-dist Neuronscited by 40
  36. 2020Lagrangian Decomposition for Neural Network Verificationcited by 35
  37. 2022LOT: Layer-wise Orthogonal Training on Improving ℓ2 Certified Robustnesscited by 33
  38. 2021Boosting the Certified Robustness of L-infinity Distance Netscited by 24

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 6; 2025: 10; 2026: 5.

23 papers.

Sort:

By area:

  1. 2023Understanding Certified Training with Interval Bound Propagationcited by 26
  2. 2024CTBENCH: A Library and Benchmark for Certified Trainingcited by 14
  3. 2023Expressivity of ReLU-Networks under Convex Relaxationscited by 8
  4. 2025Adversarial Attacks Leverage Interference Between Features in Superpositioncited by 8
  5. 2024Gaussian Loss Smoothing Enables Certified Training with Tight Convex Relaxationscited by 4
  6. 2025Faster Verified Explanations for Neural Networkscited by 4
  7. 2024On Using Certified Training towards Empirical Robustnesscited by 3
  8. 2024Expressiveness of Multi-Neuron Convex Relaxations in Neural Network Certificationcited by 2
  9. 2025BEAVER: An Efficient Deterministic LLM Verifiercited by 2
  10. 2024Towards Generalized Certified Robustness with Multi-Norm Trainingcited by 1
  11. 2024Average Certified Radius is a Poor Metric for Randomized Smoothingcited by 1
  12. 2025A Scalable Approach to Probabilistic Neuro-Symbolic Robustness Verificationcited by 1
11 more
  1. 2025On the Security Risks of ML-based Malware Detection Systems: A Surveycited by 1
  2. 2025Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exitscited by 1
  3. 2026Learning Better Certified Models from Empirically-Robust Teacherscited by 1
  4. 2025Support is All You Need for Certified VAE Trainingcited by 0
  5. 2025Compression Aware Certified Trainingcited by 0
  6. 2025Lipschitz-aware Linearity Grafting for Certified Robustnesscited by 0
  7. 2025Dual Randomized Smoothing: Beyond Global Noise Variancecited by 0
  8. 2026Rethinking Evaluation Paradigms in IBP-based Certified Trainingcited by 0
  9. 2026Hybrid Robustness Verification for Spatio-Temporal Neural Networkscited by 0
  10. 2026Improving Certified Robustness via Adversarial Distillationcited by 0
  11. 2026Certified Training for Convolutional Perturbationscited by 0