mlsanityData to 7 Oct 2026

CTBENCH: A Library and Benchmark for Certified Training

arXiv 2406.04848 ↗First posted 7 Jun 2024cs.LG cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Training certifiably robust neural networks is an important but challenging task. While many algorithms for (deterministic) certified training have been proposed, they are often evaluated on different training schedules, certification methods, and systematically under-tuned hyperparameters, making it difficult to compare their performance. To address this challenge, we introduce CTBench, a unified library and a high-quality benchmark for certified training that evaluates all algorithms under fair settings and systematically tuned hyperparameters. We show that (1) almost all algorithms in CTBench surpass the corresponding reported performance in literature in the magnitude of algorithmic improvements, thus establishing new state-of-the-art, and (2) the claimed advantage of recent algorithms drops significantly when we enhance the outdated baselines with a fair training schedule, a fair certification method and well-tuned hyperparameters. Based on CTBench, we provide new insights into the current state of certified training, including (1) certified models have less fragmented loss surface, (2) certified models share many mistakes, (3) certified models have more sparse activations, (4) reducing regularization cleverly is crucial for certified training especially for large radii and (5) certified training has the potential to improve out-of-distribution generalization. We are confident that CTBench will serve as a benchmark and testbed for future research in certified training.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 36 of its 58 references that are in the map. First posted on arXiv 7 Jun 2024.

Builds on

36 of 58 references.

References · 36

  1. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2019Benchmarking Neural Network Robustness to Common Corruptions and Perturbationscited by 2,857
  7. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  8. 2019Certified Adversarial Robustness via Randomized Smoothingcited by 1,324
  9. 2018Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examplescited by 1,259
  10. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  11. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  12. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
24 more
  1. 2020On Adaptive Attacks to Adversarial Example Defensescited by 547
  2. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 439
  3. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 322
  4. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
  5. 2020Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifierscited by 183
  6. 2018Scaling provable adversarial defensescited by 141
  7. 2019Branch and Bound for Piecewise Linear Neural Network Verificationcited by 138
  8. 2019MNIST-C: A Robustness Benchmark for Computer Visioncited by 137
  9. 2019Adversarial Examples Are a Natural Consequence of Test Error in Noisecited by 114
  10. 2022General Cutting Planes for Bound-Propagation-Based Neural Network Verificationcited by 102
  11. 2020SoK: Certified Robustness for Deep Neural Networkscited by 100
  12. 2022Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Boundcited by 92
  13. 2017How hard can it be? Estimating the difficulty of visual search in an imagecited by 52
  14. 2022Certified Training: Small Boxes are All You Needcited by 51
  15. 2021Fast Certified Robust Training with Short Warmupcited by 45
  16. 2021Rethinking "Batch" in BatchNormcited by 44
  17. 2021Certified Patch Robustness via Smoothed Vision Transformerscited by 44
  18. 2023Understanding Certified Training with Interval Bound Propagationcited by 26
  19. 2023Expressive Losses for Verified Robustness via Convex Combinationscited by 23
  20. 2022IBP Regularization for Verified Adversarial Robustness via Branch-and-Boundcited by 21
  21. 2021On the Paradox of Certified Trainingcited by 12
  22. 2023Expressivity of ReLU-Networks under Convex Relaxationscited by 8
  23. 2023Generating Less Certain Adversarial Examples Improves Robust Generalizationcited by 3
  24. 2023Efficient Symbolic Reasoning for Neural-Network Verificationcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 4; 2026: 4.

13 papers.

Sort:

By area:

  1. 2024On Using Certified Training towards Empirical Robustnesscited by 3
  2. 2024Expressiveness of Multi-Neuron Convex Relaxations in Neural Network Certificationcited by 2
  3. 2024Towards Generalized Certified Robustness with Multi-Norm Trainingcited by 1
  4. 2024Average Certified Radius is a Poor Metric for Randomized Smoothingcited by 1
  5. 2025On the Security Risks of ML-based Malware Detection Systems: A Surveycited by 1
  6. 2025Bridging Efficiency and Safety: Formal Verification of Neural Networks with Early Exitscited by 1
  7. 2026Learning Better Certified Models from Empirically-Robust Teacherscited by 1
  8. 2024MIBP-Cert: Certified Training against Data Perturbations with Mixed-Integer Bilinear Programscited by 0
  9. 2025Lipschitz-aware Linearity Grafting for Certified Robustnesscited by 0
  10. 2025Dual Randomized Smoothing: Beyond Global Noise Variancecited by 0
  11. 2026Rethinking Evaluation Paradigms in IBP-based Certified Trainingcited by 0
  12. 2026Improving Certified Robustness via Adversarial Distillationcited by 0
1 more
  1. 2026Certified Training for Convolutional Perturbationscited by 0