# FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions

arXiv: https://arxiv.org/abs/2112.13544 · HTML: https://mlsanity.com/paper/2112.13544/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 29 references that are in the map.
- First posted on arXiv 27 Dec 2021.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Understanding Deep Neural Networks with Rectified Linear Units](https://mlsanity.com/paper/1611.01491/) (2016)
- [Activation Functions: Comparison of trends in Practice and Research for Deep Learning](https://mlsanity.com/paper/1811.03378/) (2018)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks](https://mlsanity.com/paper/1906.01017/) (2019)
- [Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead](https://mlsanity.com/paper/2101.02559/) (2021)
- [FT-ClipAct: Resilience Analysis of Deep Neural Networks and Improving their Fault Tolerance using Clipped Activation](https://mlsanity.com/paper/1912.00941/) (2019)
- [HarDNN: Feature Map Vulnerability Evaluation in CNNs](https://mlsanity.com/paper/2002.09786/) (2020)
- [FAT: Training Neural Networks for Reliable Inference Under Hardware Faults](https://mlsanity.com/paper/2011.05873/) (2020)
- [Making Convolutions Resilient via Algorithm-Based Error Detection Techniques](https://mlsanity.com/paper/2006.04984/) (2020)
- [Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs](https://mlsanity.com/paper/2104.09455/) (2021)
- [When Neurons Fail](https://mlsanity.com/paper/1706.08884/) (2017)

## Built on by (most cited first, 12 of 12)

- [A Systematic Literature Review on Hardware Reliability Assessment Methods for Deep Neural Networks](https://mlsanity.com/paper/2305.05750/) (2023)
- [Evaluating Single Event Upsets in Deep Neural Networks for Semantic Segmentation: an embedded system perspective](https://mlsanity.com/paper/2412.03630/) (2024)
- [FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention](https://mlsanity.com/paper/2504.02211/) (2025)
- [DeepVigor: Vulnerability Value Ranges and Factors for DNNs' Reliability Assessment](https://mlsanity.com/paper/2303.06931/) (2023)
- [Resilience of Deep Learning applications: a systematic literature review of analysis and hardening techniques](https://mlsanity.com/paper/2309.16733/) (2023)
- [MRFI: An Open Source Multi-Resolution Fault Injection Framework for Neural Network Processing](https://mlsanity.com/paper/2306.11758/) (2023)
- [Cost-Effective Fault Tolerance for CNNs Using Parameter Vulnerability Based Hardening and Pruning](https://mlsanity.com/paper/2405.10658/) (2024)
- [Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance](https://mlsanity.com/paper/2308.08230/) (2023)
- [ProAct: Progressive Training for Hybrid Clipped Activation Function to Enhance Resilience of DNNs](https://mlsanity.com/paper/2406.06313/) (2024)
- [ZOBNN: Zero-Overhead Dependable Design of Binary Neural Networks with Deliberately Quantized Parameters](https://mlsanity.com/paper/2407.04964/) (2024)
- [Enhancing Neural Network Robustness Against Fault Injection Through Non-linear Weight Transformations](https://mlsanity.com/paper/2411.19027/) (2024)
- [TreeFI: Value-Aware Statistical Fault Injection for Deep Neural Networks](https://mlsanity.com/paper/2609.04912/) (2026)

## Abstract (arXiv)

Deep neural networks (DNNs) are increasingly being deployed in safety-critical systems such as personal healthcare devices and self-driving cars. In such DNN-based systems, error resilience is a top priority since faults in DNN inference could lead to mispredictions and safety hazards. For latency-critical DNN inference on resource-constrained edge devices, it is nontrivial to apply conventional redundancy-based fault tolerance techniques. In this paper, we propose FitAct, a low-cost approach to enhance the error resilience of DNNs by deploying fine-grained post-trainable activation functions. The main idea is to precisely bound the activation value of each individual neuron via neuron-wise bounded activation functions so that it could prevent fault propagation in the network. To avoid complex DNN model re-training, we propose to decouple the accuracy training and resilience training and develop a lightweight post-training phase to learn these activation functions with precise bound values. Experimental results on widely used DNN models such as AlexNet, VGG16, and ResNet50 demonstrate that FitAct outperforms state-of-the-art studies such as Clip-Act and Ranger in enhancing the DNN error resilience for a wide range of fault rates while adding manageable runtime and memory space overheads.