# Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead

arXiv: https://arxiv.org/abs/2205.14420 · HTML: https://mlsanity.com/paper/2205.14420/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 37 references that are in the map.
- First posted on arXiv 28 May 2022.

## Builds on (most cited first, 13 of 13)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [How Does Batch Normalization Help Optimization?](https://mlsanity.com/paper/1805.11604/) (2018)
- [Patches Are All You Need?](https://mlsanity.com/paper/2201.09792/) (2022)
- [FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their Fault Tolerance using Clipped Activation](https://mlsanity.com/paper/1912.00941/) (2019)
- [FAT: Training Neural Networks for Reliable Inference Under Hardware Faults](https://mlsanity.com/paper/2011.05873/) (2020)
- [Making Convolutions Resilient via Algorithm-Based Error Detection Techniques](https://mlsanity.com/paper/2006.04984/) (2020)
- [A Low-cost Fault Corrector for Deep Neural Networks through Range Restriction](https://mlsanity.com/paper/2003.13874/) (2020)

## Built on by (most cited first, 3 of 3)

- [DeepAxe: A Framework for Exploration of Approximation and Reliability Trade-offs in DNN Accelerators](https://mlsanity.com/paper/2303.08226/) (2023)
- [Cost-Effective Fault Tolerance for CNNs Using Parameter Vulnerability Based Hardening and Pruning](https://mlsanity.com/paper/2405.10658/) (2024)
- [ProAct: Progressive Training for Hybrid Clipped Activation Function to Enhance Resilience of DNNs](https://mlsanity.com/paper/2406.06313/) (2024)

## Abstract (arXiv)

Deep Neural Networks (DNNs) enable a wide series of technological advancements, ranging from clinical imaging, to predictive industrial maintenance and autonomous driving. However, recent findings indicate that transient hardware faults may corrupt the models prediction dramatically. For instance, the radiation-induced misprediction probability can be so high to impede a safe deployment of DNNs models at scale, urging the need for efficient and effective hardening solutions. In this work, we propose to tackle the reliability issue both at training and model design time. First, we show that vanilla models are highly affected by transient faults, that can induce a performances drop up to 37%. Hence, we provide three zero-overhead solutions, based on DNN re-design and re-train, that can improve DNNs reliability to transient faults up to one order of magnitude. We complement our work with extensive ablation studies to quantify the gain in performances of each hardening component.