# Statistical Modeling of Soft Error Influence on Neural Networks

arXiv: https://arxiv.org/abs/2210.05876 · HTML: https://mlsanity.com/paper/2210.05876/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 49 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 12 of 12)

- [Taxonomy of Real Faults in Deep Learning Systems](https://mlsanity.com/paper/1910.11015/) (2019)
- [Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead](https://mlsanity.com/paper/2101.02559/) (2021)
- [TRADI: Tracking deep neural network weight distributions for uncertainty estimation](https://mlsanity.com/paper/1912.11316/) (2019)
- [Analyzing and Mitigating the Impact of Permanent Faults on a Systolic Array Based Neural Network Accelerator](https://mlsanity.com/paper/1802.04657/) (2018)
- [TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications](https://mlsanity.com/paper/2004.01743/) (2020)
- [AI Accelerator Survey and Trends](https://mlsanity.com/paper/2109.08957/) (2021)
- [FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture](https://mlsanity.com/paper/2003.10375/) (2020)
- [Efficient Error-Tolerant Quantized Neural Network Accelerators](https://mlsanity.com/paper/1912.07394/) (2019)
- [R2F: A Remote Retraining Framework for AIoT Processors with Computing Errors](https://mlsanity.com/paper/2107.03096/) (2021)
- [Winograd Convolution: A Perspective from Fault Tolerance](https://mlsanity.com/paper/2202.08675/) (2022)
- [HyCA: A Hybrid Computing Architecture for Fault Tolerant Deep Learning](https://mlsanity.com/paper/2106.04772/) (2021)
- [Fault-Tolerant Deep Learning: A Hierarchical Perspective](https://mlsanity.com/paper/2204.01942/) (2022)

## Built on by (most cited first, 2 of 2)

- [MRFI: An Open Source Multi-Resolution Fault Injection Framework for Neural Network Processing](https://mlsanity.com/paper/2306.11758/) (2023)
- [Analysis of LLM Vulnerability to GPU Soft Errors: An Instruction-Level Fault Injection Study](https://mlsanity.com/paper/2601.19912/) (2025)

## Abstract (arXiv)

Soft errors in large VLSI circuits pose dramatic influence on computing- and memory-intensive neural network (NN) processing. Understanding the influence of soft errors on NNs is critical to protect against soft errors for reliable NN processing. Prior work mainly rely on fault simulation to analyze the influence of soft errors on NN processing. They are accurate but usually specific to limited configurations of errors and NN models due to the prohibitively slow simulation speed especially for large NN models and datasets. With the observation that the influence of soft errors propagates across a large number of neurons and accumulates as well, we propose to characterize the soft error induced data disturbance on each neuron with normal distribution model according to central limit theorem and develop a series of statistical models to analyze the behavior of NN models under soft errors in general. The statistical models reveal not only the correlation between soft errors and NN model accuracy, but also how NN parameters such as quantization and architecture affect the reliability of NNs. The proposed models are compared with fault simulation and verified comprehensively. In addition, we observe that the statistical models that characterize the soft error influence can also be utilized to predict fault simulation results in many cases and we explore the use of the proposed statistical models to accelerate fault simulations of NNs. According to our experiments, the accelerated fault simulation shows almost two orders of magnitude speedup with negligible simulation accuracy loss over the baseline fault simulations.