# Cost-Effective Fault Tolerance for CNNs Using Parameter Vulnerability Based Hardening and Pruning

arXiv: https://arxiv.org/abs/2405.10658 · HTML: https://mlsanity.com/paper/2405.10658/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 17 May 2024.

## Builds on (most cited first, 13 of 13)

- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their Fault Tolerance using Clipped Activation](https://mlsanity.com/paper/1912.00941/) (2019)
- [Analyzing and Mitigating the Impact of Permanent Faults on a Systolic Array Based Neural Network Accelerator](https://mlsanity.com/paper/1802.04657/) (2018)
- [A Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networks](https://mlsanity.com/paper/2305.05750/) (2023)
- [FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks](https://mlsanity.com/paper/2003.12203/) (2020)
- [HarDNN: Feature Map Vulnerability Evaluation in CNNs](https://mlsanity.com/paper/2002.09786/) (2020)
- [FAT: Training Neural Networks for Reliable Inference Under Hardware Faults](https://mlsanity.com/paper/2011.05873/) (2020)
- [FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions](https://mlsanity.com/paper/2112.13544/) (2021)
- [DeepVigor: Vulnerability Value Ranges and Factors for DNNs' Reliability Assessment](https://mlsanity.com/paper/2303.06931/) (2023)
- [Enhancing Fault Resilience of QNNs by Selective Neuron Splitting](https://mlsanity.com/paper/2306.09973/) (2023)
- [Fault-Aware Design and Training to Enhance DNNs Reliability with Zero-Overhead](https://mlsanity.com/paper/2205.14420/) (2022)

## Built on by (most cited first, 3 of 3)

- [ProAct: Progressive Training for Hybrid Clipped Activation Function to Enhance Resilience of DNNs](https://mlsanity.com/paper/2406.06313/) (2024)
- [NAPER: Fault Protection for Real-Time Resource-Constrained Deep Neural Networks](https://mlsanity.com/paper/2504.06591/) (2025)
- [Adaptive Soft Error Protection for Neural Network Processing](https://mlsanity.com/paper/2407.19664/) (2024)

## Abstract (arXiv)

Convolutional Neural Networks (CNNs) have become integral in safety-critical applications, thus raising concerns about their fault tolerance. Conventional hardware-dependent fault tolerance methods, such as Triple Modular Redundancy (TMR), are computationally expensive, imposing a remarkable overhead on CNNs. Whereas fault tolerance techniques can be applied either at the hardware level or at the model levels, the latter provides more flexibility without sacrificing generality. This paper introduces a model-level hardening approach for CNNs by integrating error correction directly into the neural networks. The approach is hardware-agnostic and does not require any changes to the underlying accelerator device. Analyzing the vulnerability of parameters enables the duplication of selective filters/neurons so that their output channels are effectively corrected with an efficient and robust correction layer. The proposed method demonstrates fault resilience nearly equivalent to TMR-based correction but with significantly reduced overhead. Nevertheless, there exists an inherent overhead to the baseline CNNs. To tackle this issue, a cost-effective parameter vulnerability based pruning technique is proposed that outperforms the conventional pruning method, yielding smaller networks with a negligible accuracy loss. Remarkably, the hardened pruned CNNs perform up to 24% faster than the hardened un-pruned ones.