# FAT: Training Neural Networks for Reliable Inference Under Hardware Faults

arXiv: https://arxiv.org/abs/2011.05873 · HTML: https://mlsanity.com/paper/2011.05873/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Nov 2020.

## Built on by (most cited first, 15 of 15)

- [A Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networks](https://mlsanity.com/paper/2305.05750/) (2023)
- [FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions](https://mlsanity.com/paper/2112.13544/) (2021)
- [Enhancing Fault Resilience of QNNs by Selective Neuron Splitting](https://mlsanity.com/paper/2306.09973/) (2023)
- [Resilience of Deep Learning applications: a systematic literature review of analysis and hardening techniques](https://mlsanity.com/paper/2309.16733/) (2023)
- [Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead](https://mlsanity.com/paper/2205.14420/) (2022)
- [Cost-Effective Fault Tolerance for CNNs Using Parameter Vulnerability Based Hardening and Pruning](https://mlsanity.com/paper/2405.10658/) (2024)
- [FAQ: Mitigating the Impact of Faults in the Weight Memory of DNN Accelerators through Fault-Aware Quantization](https://mlsanity.com/paper/2305.12590/) (2023)
- [Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance](https://mlsanity.com/paper/2308.08230/) (2023)
- [ProAct: Progressive Training for Hybrid Clipped Activation Function to Enhance Resilience of DNNs](https://mlsanity.com/paper/2406.06313/) (2024)
- [Transient Fault Tolerant Semantic Segmentation for Autonomous Driving](https://mlsanity.com/paper/2408.16952/) (2024)
- [Regularization-based Framework for Quantization-, Fault- and Variability-Aware Training](https://mlsanity.com/paper/2503.01297/) (2025)
- [FORTALESA: Fault-Tolerant Reconfigurable Systolic Array for DNN Inference](https://mlsanity.com/paper/2503.04426/) (2025)
- [FlatENN: Train Flat for Enhanced Fault Tolerance of Quantized Deep Neural Networks](https://mlsanity.com/paper/2301.00675/) (2022)
- [Memory Faults in Activation-sparse Quantized Deep Neural Networks: Analysis and Mitigation using Sharpness-aware Training](https://mlsanity.com/paper/2406.10528/) (2024)
- [Mitigating multiple single-event upsets during deep neural network inference using fault-aware training](https://mlsanity.com/paper/2502.09374/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) are state-of-the-art algorithms for multiple applications, spanning from image classification to speech recognition. While providing excellent accuracy, they often have enormous compute and memory requirements. As a result of this, quantized neural networks (QNNs) are increasingly being adopted and deployed especially on embedded devices, thanks to their high accuracy, but also since they have significantly lower compute and memory requirements compared to their floating point equivalents. QNN deployment is also being evaluated for safety-critical applications, such as automotive, avionics, medical or industrial. These systems require functional safety, guaranteeing failure-free behaviour even in the presence of hardware faults. In general fault tolerance can be achieved by adding redundancy to the system, which further exacerbates the overall computational demands and makes it difficult to meet the power and performance requirements. In order to decrease the hardware cost for achieving functional safety, it is vital to explore domain-specific solutions which can exploit the inherent features of DNNs. In this work we present a novel methodology called fault-aware training (FAT), which includes error modeling during neural network (NN) training, to make QNNs resilient to specific fault models on the device. Our experiments show that by injecting faults in the convolutional layers during training, highly accurate convolutional neural networks (CNNs) can be trained which exhibits much better error tolerance compared to the original. Furthermore, we show that redundant systems which are built from QNNs trained with FAT achieve higher worse-case accuracy at lower hardware cost. This has been validated for numerous classification tasks including CIFAR10, GTSRB, SVHN and ImageNet.