# Formal and Practical Elements for the Certification of Machine Learning Systems

arXiv: https://arxiv.org/abs/2310.03217 · HTML: https://mlsanity.com/paper/2310.03217/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 43 references that are in the map.
- First posted on arXiv 5 Oct 2023.

## Builds on (most cited first, 11 of 11)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Reluplex: An Efficient SMT Solver for Verifying Deep Neural Networks](https://mlsanity.com/paper/1702.01135/) (2017)
- [DeepTest: Automated Testing of Deep-Neural-Network-driven Autonomous Cars](https://mlsanity.com/paper/1708.08559/) (2017)
- [Algorithms for Verifying Deep Neural Networks](https://mlsanity.com/paper/1903.06758/) (2019)
- [A Survey of Algorithms for Black-Box Safety Validation of Cyber-Physical Systems](https://mlsanity.com/paper/2005.02979/) (2020)
- [Efficient Black-box Assessment of Autonomous Vehicle Safety](https://mlsanity.com/paper/1912.03618/) (2019)
- [Correctness Verification of Neural Networks](https://mlsanity.com/paper/1906.01030/) (2019)
- [Bayesian Safety Validation for Failure Probability Estimation of Black-Box Systems](https://mlsanity.com/paper/2305.02449/) (2023)
- [Framework for Certification of AI-Based Systems](https://mlsanity.com/paper/2302.11049/) (2023)

## Built on by (most cited first, 2 of 2)

- [Towards certification: A complete statistical validation pipeline for supervised learning in industry](https://mlsanity.com/paper/2411.02075/) (2024)
- [Aircraft Collision Avoidance Systems: Technological Challenges and Solutions on the Path to Regulatory Acceptance](https://mlsanity.com/paper/2510.20916/) (2025)

## Abstract (arXiv)

Over the past decade, machine learning has demonstrated impressive results, often surpassing human capabilities in sensing tasks relevant to autonomous flight. Unlike traditional aerospace software, the parameters of machine learning models are not hand-coded nor derived from physics but learned from data. They are automatically adjusted during a training phase, and their values do not usually correspond to physical requirements. As a result, requirements cannot be directly traced to lines of code, hindering the current bottom-up aerospace certification paradigm. This paper attempts to address this gap by 1) demystifying the inner workings and processes to build machine learning models, 2) formally establishing theoretical guarantees given by those processes, and 3) complementing these formal elements with practical considerations to develop a complete certification argument for safety-critical machine learning systems. Based on a scalable statistical verifier, our proposed framework is model-agnostic and tool-independent, making it adaptable to many use cases in the industry. We demonstrate results on a widespread application in autonomous flight: vision-based landing.