# FANNet: Formal Analysis of Noise Tolerance, Training Bias and Input Sensitivity in Neural Networks

arXiv: https://arxiv.org/abs/1912.01978 · HTML: https://mlsanity.com/paper/1912.01978/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Dec 2019.

## Built on by (most cited first, 7 of 7)

- [How to Certify Machine Learning Based Safety-critical Systems? A Systematic Literature Review](https://mlsanity.com/paper/2107.12045/) (2021)
- [Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework](https://mlsanity.com/paper/2109.09829/) (2021)
- [Machine Learning Robustness: A Primer](https://mlsanity.com/paper/2404.00897/) (2024)
- [Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems](https://mlsanity.com/paper/2204.09514/) (2022)
- [Exploiting Vulnerabilities in Deep Neural Networks: Adversarial and Fault-Injection Attacks](https://mlsanity.com/paper/2105.03251/) (2021)
- [UnbiasedNets: A Dataset Diversification Framework for Robustness Bias Alleviation in Neural Networks](https://mlsanity.com/paper/2302.12538/) (2023)
- [Scaling Model Checking for DNN Analysis via State-Space Reduction and Input Segmentation (Extended Version)](https://mlsanity.com/paper/2306.17323/) (2023)

## Abstract (arXiv)

With a constant improvement in the network architectures and training methodologies, Neural Networks (NNs) are increasingly being deployed in real-world Machine Learning systems. However, despite their impressive performance on "known inputs", these NNs can fail absurdly on the "unseen inputs", especially if these real-time inputs deviate from the training dataset distributions, or contain certain types of input noise. This indicates the low noise tolerance of NNs, which is a major reason for the recent increase of adversarial attacks. This is a serious concern, particularly for safety-critical applications, where inaccurate results lead to dire consequences. We propose a novel methodology that leverages model checking for the Formal Analysis of Neural Network (FANNet) under different input noise ranges. Our methodology allows us to rigorously analyze the noise tolerance of NNs, their input node sensitivity, and the effects of training bias on their performance, e.g., in terms of classification accuracy. For evaluation, we use a feed-forward fully-connected NN architecture trained for the Leukemia classification. Our experimental results show ± 11% noise tolerance for the given trained network, identify the most sensitive input nodes, and confirm the biasness of the available training dataset.