# Towards Practical Verification of Machine Learning: The Case of Computer Vision Systems

arXiv: https://arxiv.org/abs/1712.01785 · HTML: https://mlsanity.com/paper/1712.01785/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Dec 2017.

## Built on by (most cited first, 19 of 19)

- [DriveSuprim: Towards Precise Trajectory Selection for End-to-End Planning](https://mlsanity.com/paper/2506.06659/) (2025)
- [Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead](https://mlsanity.com/paper/2101.02559/) (2021)
- [The future of human-AI collaboration: a taxonomy of design knowledge for hybrid intelligence systems](https://mlsanity.com/paper/2105.03354/) (2021)
- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [A review on data-driven constitutive laws for solids](https://mlsanity.com/paper/2405.03658/) (2024)
- [Taxonomy of Machine Learning Safety: A Survey and Primer](https://mlsanity.com/paper/2106.04823/) (2021)
- [GSmooth: Certified Robustness against Semantic Transformations via Generalized Randomized Smoothing](https://mlsanity.com/paper/2206.04310/) (2022)
- [Robustness Certification for Point Cloud Models](https://mlsanity.com/paper/2103.16652/) (2021)
- [Towards Verifying the Geometric Robustness of Large-scale Neural Networks](https://mlsanity.com/paper/2301.12456/) (2023)
- [Shared Certificates for Neural Network Verification](https://mlsanity.com/paper/2109.00542/) (2021)
- [Responsible and Regulatory Conform Machine Learning for Medicine: A Survey of Challenges and Solutions](https://mlsanity.com/paper/2107.09546/) (2021)
- [Verification of Geometric Robustness of Neural Networks via Piecewise Linear Approximation and Lipschitz Optimisation](https://mlsanity.com/paper/2408.13140/) (2024)
- [Graph Neural Backdoor: Fundamentals, Methodologies, Applications, and Future Directions](https://mlsanity.com/paper/2406.10573/) (2024)
- [Pixel-wise Smoothing for Certified Robustness against Camera Motion Perturbations](https://mlsanity.com/paper/2309.13150/) (2023)
- [Embodied Learning for Lifelong Visual Perception](https://mlsanity.com/paper/2112.14084/) (2021)
- [Enhancing Visual Perception in Novel Environments via Incremental Data Augmentation Based on Style Transfer](https://mlsanity.com/paper/2309.08851/) (2023)
- [General Lipschitz: Certified Robustness Against Resolvable Semantic Transformations via Transformation-Dependent Randomized Smoothing](https://mlsanity.com/paper/2309.16710/) (2023)
- [Towards Improving Robustness Against Common Corruptions in Object Detectors Using Adversarial Contrastive Learning](https://mlsanity.com/paper/2311.07928/) (2023)
- [Transform-Dependent Adversarial Attacks](https://mlsanity.com/paper/2406.08443/) (2024)

## Abstract (arXiv)

Due to the increasing usage of machine learning (ML) techniques in security- and safety-critical domains, such as autonomous systems and medical diagnosis, ensuring correct behavior of ML systems, especially for different corner cases, is of growing importance. In this paper, we propose a generic framework for evaluating security and robustness of ML systems using different real-world safety properties. We further design, implement and evaluate VeriVis, a scalable methodology that can verify a diverse set of safety properties for state-of-the-art computer vision systems with only blackbox access. VeriVis leverage different input space reduction techniques for efficient verification of different safety properties. VeriVis is able to find thousands of safety violations in fifteen state-of-the-art computer vision systems including ten Deep Neural Networks (DNNs) such as Inception-v3 and Nvidia's Dave self-driving system with thousands of neurons as well as five commercial third-party vision APIs including Google vision and Clarifai for twelve different safety properties. Furthermore, VeriVis can successfully verify local safety properties, on average, for around 31.7% of the test images. VeriVis finds up to 64.8x more violations than existing gradient-based methods that, unlike VeriVis, cannot ensure non-existence of any violations. Finally, we show that retraining using the safety violations detected by VeriVis can reduce the average number of violations up to 60.2%.