mlsanityData to 14 Oct 2026

First Three Years of the International Verification of Neural Networks Competition (VNN-COMP)

arXiv 2301.05815 ↗First posted 14 Jan 2023cs.LG cs.AI cs.SEcited by 58

Not on the map. Why?

Abstract (arXiv)

This paper presents a summary and meta-analysis of the first three iterations of the annual International Verification of Neural Networks Competition (VNN-COMP) held in 2020, 2021, and 2022. In the VNN-COMP, participants submit software tools that analyze whether given neural networks satisfy specifications describing their input-output behavior. These neural networks and specifications cover a variety of problem classes and tasks, corresponding to safety and robustness properties in image classification, neural control, reinforcement learning, and autonomous systems. We summarize the key processes, rules, and results, present trends observed over the last three years, and provide an outlook into possible future developments.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 36 of its 70 references that are in the map. First posted on arXiv 14 Jan 2023.

Builds on

36 of 70 references.

References · 36

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,364
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
  4. 2013Intriguing properties of neural networkscited by 5,155
  5. 2016End to End Learning for Self-Driving Carscited by 1,009
  6. 2017Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networkscited by 744
  7. 2018Efficient Neural Network Robustness Certification with General Activation Functionscited by 445
  8. 2018On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Modelscited by 328
  9. 2016Safety Verification of Deep Neural Networkscited by 263
  10. 2017Formal Verification of Piece-Wise Linear Feed-Forward Neural Networkscited by 242
  11. 2019Algorithms for Verifying Deep Neural Networkscited by 221
  12. 2019Towards Stable and Efficient Training of Verifiably Robust Neural Networkscited by 205
24 more
  1. 2020Fast and Complete: Enabling Complete Neural Network Verification with Rapid and Massively Parallel Incomplete Verifierscited by 187
  2. 2019Branch and Bound for Piecewise Linear Neural Network Verificationcited by 138
  3. 2020NNV: The Neural Network Verification Tool for Deep Neural Networks and Learning-Enabled Cyber-Physical Systemscited by 128
  4. 2022General Cutting Planes for Bound-Propagation-Based Neural Network Verificationcited by 103
  5. 2022Complete Verification via Multi-Neuron Relaxation Guided Branch-and-Boundcited by 92
  6. 2017Output Reachable Set Estimation and Verification for Multi-Layer Neural Networkscited by 80
  7. 2020Verification of Deep Convolutional Neural Networks Using ImageStarscited by 75
  8. 2021The Second International Verification of Neural Networks Competition (VNN-COMP 2021): Summary and Resultscited by 71
  9. 2022Certified Training: Small Boxes are All You Needcited by 52
  10. 2021Fast Certified Robust Training with Short Warmupcited by 46
  11. 2019Neural Network Branching for Neural Network Verificationcited by 45
  12. 2021Improved Branch and Bound for Neural Network Verification via Lagrangian Decompositioncited by 44
  13. 2020Scaling Polyhedral Neural Network Verification on GPUscited by 43
  14. 2022The Third International Verification of Neural Networks Competition (VNN-COMP 2022): Summary and Resultscited by 39
  15. 2020Reachable Polyhedral Marching (RPM): A Safety Verification Algorithm for Robotic Systems with Deep Neural Network Componentscited by 36
  16. 2020Lagrangian Decomposition for Neural Network Verificationcited by 35
  17. 2021DNNV: A Framework for Deep Neural Network Verificationcited by 23
  18. 2022Neural Network Verification with Proof Productioncited by 22
  19. 2022IBP Regularization for Verified Adversarial Robustness via Branch-and-Boundcited by 21
  20. 2021Scaling the Convex Barrier with Sparse Dual Algorithmscited by 10
  21. 2021Shared Certificates for Neural Network Verificationcited by 10
  22. 2019AReN: Assured ReLU NN Architecture for Model Predictive Control of LTI Systemscited by 9
  23. 2020Debona: Decoupled Boundary Network Analysis for Tighter Bounds and Faster Adversarial Robustness Proofscited by 8
  24. 2020PEREGRiNN: Penalized-Relaxation Greedy Neural Network Verifiercited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 10; 2024: 17; 2025: 14; 2026: 17.

58 papers.

Sort:

By area:

  1. 2023Provably Bounding Neural Network Preimagescited by 29
  2. 2023Understanding Certified Training with Interval Bound Propagationcited by 26
  3. 2023A DPLL(T) Framework for Verifying Deep Neural Networkscited by 24
  4. 2024Local vs. Global Interpretability: A Computational Complexity Perspectivecited by 23
  5. 2024NLP Verification: Towards a General Methodology for Certifying Robustnesscited by 16
  6. 2024Harnessing Neuron Stability to Improve DNN Verificationcited by 13
  7. 2025Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanationscited by 13
  8. 2025Neural Network Verification is a Programming Language Challengecited by 12
  9. 2025SDP-CROWN: Efficient Bound Propagation for Neural Network Verification with Tightness of Semidefinite Programmingcited by 12
  10. 2023TAPS: Connecting Certified and Adversarial Trainingcited by 11
  11. 2026Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guaranteescited by 11
  12. 2024Detecting Brittle Decisions for Free: Leveraging Margin Consistency in Deep Robust Classifierscited by 10
38 more
  1. 2024Robustness Assessment of a Runway Object Classifier for Safe Aircraft Taxiingcited by 8
  2. 2024Provably Safe Neural Network Controllers via Differential Dynamic Logiccited by 7
  3. 2024A Certified Proof Checker for Deep Neural Network Verification in Imandracited by 7
  4. 2024ModelVerification.jl: a Comprehensive Toolbox for Formally Verifying Deep Neural Networkscited by 7
  5. 2023When to Trust AI: Advances and Challenges for Certification of Neural Networkscited by 6
  6. 2024Set-Based Training for Neural Network Verificationcited by 6
  7. 2025Out of the Shadows: Exploring a Latent Space for Neural Network Verificationcited by 6
  8. 2024Relational DNN Verification With Cross Executional Bound Refinementcited by 5
  9. 2024SoundnessBench: A Soundness Benchmark for Neural Network Verifierscited by 5
  10. 2025Abstraction-Based Proof Production in Formal Verification of Neural Networkscited by 5
  11. 2023GPU-Accelerated Verification of Machine Learning Models for Power Systemscited by 4
  12. 2024Gaussian Loss Smoothing Enables Certified Training with Tight Convex Relaxationscited by 4
  13. 2024Revisiting Differential Verification: Equivalence Verification with Confidencecited by 4
  14. 2025Proof Minimization in Neural Network Verificationcited by 4
  15. 2025Faster Verified Explanations for Neural Networkscited by 4
  16. 2023DelBugV: Delta-Debugging Neural Network Verifierscited by 3
  17. 2024Certified Robust Invariant Polytope Training in Neural Controlled ODEscited by 3
  18. 2025PICID: Proof-Driven Clause Learning in Neural Network Verificationcited by 3
  19. 2026Robustness Verification of Polynomial Neural Networkscited by 3
  20. 2024Logic-Based Explainability: Past, Present & Futurecited by 2
  21. 2024Achieving the Tightest Relaxation of Sigmoids for Formal Verificationcited by 2
  22. 2024Verification of Neural Networks against Convolutional Perturbations via Parameterised Kernelscited by 2
  23. 2026Incremental Neural Network Verification via Learned Conflictscited by 2
  24. 2023Is Certifying ℓp Robustness Still Worthwhile?cited by 1
  25. 2023The Pros and Cons of Adversarial Robustnesscited by 1
  26. 2025A General Framework for Property-Driven Machine Learningcited by 1
  27. 2026Cascading Robustness Verification: Toward Efficient Model-Agnostic Certificationcited by 1
  28. 2026IoUCert: Robustness Verification for Anchor-based Object Detectorscited by 1
  29. 2026VNN-LIB 2.0: Rigorous Foundations for Neural Network Verificationcited by 1
  30. 2023Syntactic vs Semantic Linear Abstraction and Refinement of Neural Networkscited by 0
  31. 2025Language Models That Walk the Talk: A Framework for Formal Fairness Certificatescited by 0
  32. 2025The CAISAR Platform: Extending the Reach of Machine Learning Specification and Verificationcited by 0
  33. 2025Of Good Demons and Bad Angels: Guaranteeing Safe Control under Finite Precisioncited by 0
  34. 2025Rigorous Feature Importance Scores based on Shapley Value and Banzhaf Indexcited by 0
  35. 2025Aircraft Collision Avoidance Systems: Technological Challenges and Solutions on the Path to Regulatory Acceptancecited by 0
  36. 2026Certified Learning under Distribution Shift: Sound Verification and Identifiable Structurecited by 0
  37. 2026Adversarial Robustness of Time-Series Classification for Crystal Collimator Alignmentcited by 0
  38. 2026The Cost of Relaxation: Evaluating the Error in Convex Neural Network Verificationcited by 0

50 most cited of 58 All 58 (JSON)