mlsanityData to 7 Oct 2026

The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Results

arXiv 2212.10376 ↗First posted 20 Dec 2022cs.LG cs.AI cs.SEcited by 39

Not on the map. Why?

Abstract (arXiv)

This report summarizes the 3rd International Verification of Neural Networks Competition (VNN-COMP 2022), held as a part of the 5th Workshop on Formal Methods for ML-Enabled Autonomous Systems (FoMLAS), which was collocated with the 34th International Conference on Computer-Aided Verification (CAV). VNN-COMP is held annually to facilitate the fair and objective comparison of state-of-the-art neural network verification tools, encourage the standardization of tool interfaces, and bring together the neural network verification community. To this end, standardized formats for networks (ONNX) and specification (VNN-LIB) were defined, tools were evaluated on equal-cost hardware (using an automatic evaluation pipeline based on AWS instances), and tool parameters were chosen by the participants before the final test sets were made public. In the 2022 iteration, 11 teams participated on a diverse set of 12 scored benchmarks. This report summarizes the rules, benchmarks, participating tools, results, and lessons learned from this iteration of this competition.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Builds on 29 of its 53 references that are in the map. First posted on arXiv 20 Dec 2022.

Builds on

29 of 53 references.

References · 29

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  6. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 739
  7. 2017Provable defenses against adversarial examples via the convex outer adversarial polytopecited by 564
  8. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 204
  9. 2017The Case for Learned Index Structurescited by 188
  10. 2020Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifierscited by 183
  11. 2022General Cutting Planes for Bound-Propagation-Based Neural Network Verificationcited by 102
  12. 2022Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Boundcited by 92
17 more
  1. 2020Verification of Deep Convolutional Neural Networks Using ImageStarscited by 75
  2. 2019Neural Network Branching for Neural Network Verificationcited by 45
  3. 2021Improved Branch and Bound for Neural Network Verification via Lagrangian Decompositioncited by 44
  4. 2020Parallelization Techniques for Verifying Neural Networkscited by 36
  5. 2020Lagrangian Decomposition for Neural Network Verificationcited by 35
  6. 2020Abstraction based Output Range Analysis for Neural Networkscited by 34
  7. 2022Efficient Neural Network Analysis with Sum-of-Infeasibilitiescited by 24
  8. 2021DNNV: A Framework for Deep Neural Network Verificationcited by 23
  9. 2022Scalable Verification of GNN-based Job Schedulerscited by 18
  10. 2022Toward Certified Robustness Against Real-World Distribution Shiftscited by 16
  11. 2021Generating Adversarial Examples with Graph Neural Networkscited by 12
  12. 2021Learning Density Distribution of Reachable States for Autonomous Systemscited by 11
  13. 2021Scaling the Convex Barrier with Sparse Dual Algorithmscited by 10
  14. 2020Debona: Decoupled Boundary Network Analysis for Tighter Bounds and Faster Adversarial Robustness Proofscited by 8
  15. 2021Fast BATLLNN: Fast Box Analysis of Two-Level Lattice Neural Networkscited by 8
  16. 2021Neural Network Branch-and-Bound for Neural Network Verificationcited by 6
  17. 2022Case Studies for Computing Density of Reachable States for Safe Autonomous Motion Planningcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 15; 2024: 11; 2025: 7; 2026: 6.

39 papers.

Sort:

  1. 2023First Three Years of the International Verification of Neural Networks Competition (VNN-COMP)cited by 58
  2. 2024Scalable Neural Network Verification with Branch-and-bound Inferred Cutting Planescited by 33
  3. 2023Understanding Certified Training with Interval Bound Propagationcited by 26
  4. 2023A DPLL(T) Framework for Verifying Deep Neural Networkscited by 23
  5. 2024Harnessing Neuron Stability to Improve DNN Verificationcited by 13
  6. 2023TAPS: Connecting Certified and Adversarial Trainingcited by 11
  7. 2026Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guaranteescited by 10
  8. 2023Asymmetric Certified Robustness via Feature-Convex Neural Networkscited by 8
  9. 2024PREMAP: A Unifying PREiMage APproximation Framework for Neural Networkscited by 8
  10. 2024ModelVerification.jl: a Comprehensive Toolbox for Formally Verifying Deep Neural Networkscited by 7
  11. 2023ANTONIO: Towards a Systematic Method of Generating NLP Benchmarks for Verificationcited by 6
  12. 2023DeepBern-Nets: Taming the Complexity of Certifying Neural Networks using Bernstein Polynomial Activations and Precise Bound Propagationcited by 5
27 more
  1. 2024SoundnessBench: A Soundness Benchmark for Neural Network Verifierscited by 5
  2. 2025Position: Certified Robustness Does Not (Yet) Imply Model Securitycited by 4
  3. 2023A Tale of Two Approximations: Tightening Over-Approximation for DNN Robustness Verification via Under-Approximationcited by 2
  4. 2023Robustness Verification of Deep Neural Networks using Star-Based Reachability Analysis with Variable-Length Time Series Inputcited by 2
  5. 2024DEM: A Method for Certifying Deep Neural Network Classifier Outputs in Aerospacecited by 2
  6. 2024Scalable Exact Verification of Optimization Proxies for Large-Scale Optimal Power Flowcited by 2
  7. 2024Monitizer: Automating Design and Evaluation of Neural Network Monitorscited by 2
  8. 2024Verification of Neural Networks against Convolutional Perturbations via Parameterised Kernelscited by 2
  9. 2025Adaptive Branch-and-Bound Tree Exploration for Neural Network Verificationcited by 2
  10. 2025Efficient Preimage Approximation for Neural Network Certificationcited by 2
  11. 2025Floating-Point Neural Network Verification at the Software Levelcited by 2
  12. 2023Expediting Neural Network Verification via Network Reductioncited by 1
  13. 2023Comparing Differentiable Logics for Learning Systems: A Research Previewcited by 1
  14. 2023The Pros and Cons of Adversarial Robustnesscited by 1
  15. 2025A General Framework for Property-Driven Machine Learningcited by 1
  16. 2025Fast and Flexible Robustness Certificates for Semantic Segmentationcited by 1
  17. 2023CGDTest: A Constrained Gradient Descent Algorithm for Testing Neural Networkscited by 0
  18. 2023Verifying Global Neural Network Specifications using Hyperpropertiescited by 0
  19. 2023Adaptive Certified Training: Towards Better Accuracy-Robustness Tradeoffscited by 0
  20. 2024Towards General Robustness Verification of MaxPool-based Convolutional Neural Networks via Tightening Linear Approximationcited by 0
  21. 2024Creating a Formally Verified Neural Network for Autonomous Navigation: An Experience Reportcited by 0
  22. 2025Efficient Neural Network Verification via Order Leading Exploration of Branch-and-Bound Treescited by 0
  23. 2026Analyzing Symbolic Properties for DRL Agents in Systems and Networkingcited by 0
  24. 2026Scaling Neural Network Verification with Tensor Parallelism and Fully Sharded Data Parallelismcited by 0
  25. 2026Shifting-based Optimizable Linear Relaxations for General Activation Functionscited by 0
  26. 2026Certified but Private: Scalable Zero-Knowledge Proofs for Neural Network Guaranteescited by 0
  27. 2026Reachability-Based Formal Verification of Graph Neural Networks with Node and Edge Featurescited by 0