# Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs

arXiv: https://arxiv.org/abs/2104.09455 · HTML: https://mlsanity.com/paper/2104.09455/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 90 references that are in the map.
- First posted on arXiv 19 Apr 2021.

## Builds on (most cited first, 25 of 25)

- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [TVM: An Automated End-to-End Optimizing Compiler for Deep Learning](https://mlsanity.com/paper/1802.04799/) (2018)
- [Online Model Distillation for Efficient Video Inference](https://mlsanity.com/paper/1812.02699/) (2018)
- [Focus: Querying Large Video Datasets with Low Latency and Low Cost](https://mlsanity.com/paper/1801.03493/) (2018)
- [BlazeIt: Optimizing Declarative Aggregation and Limit Queries for Neural Network-Based Video Analytics](https://mlsanity.com/paper/1805.01046/) (2018)
- [NoScope: Optimizing Neural Network Queries over Video at Scale](https://mlsanity.com/paper/1703.02529/) (2017)
- [EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAM](https://mlsanity.com/paper/1910.05340/) (2019)
- [Silent Data Corruptions at Scale](https://mlsanity.com/paper/2102.11245/) (2021)
- [TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications](https://mlsanity.com/paper/2004.01743/) (2020)
- [FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks](https://mlsanity.com/paper/2003.12203/) (2020)
- [ThUnderVolt: Enabling Aggressive Voltage Underscaling and Timing Error Resilience for Energy Efficient Deep Neural Network Accelerators](https://mlsanity.com/paper/1802.03806/) (2018)
- [Fast Video Classification via Adaptive Cascading of Deep Models](https://mlsanity.com/paper/1611.06453/) (2016)
- [HarDNN: Feature Map Vulnerability Evaluation in CNNs](https://mlsanity.com/paper/2002.09786/) (2020)
- [Dissecting the NVidia Turing T4 GPU via Microbenchmarking](https://mlsanity.com/paper/1903.07486/) (2019)
- [Making Convolutions Resilient via Algorithm-Based Error Detection Techniques](https://mlsanity.com/paper/2006.04984/) (2020)
- [DeepDyve: Dynamic Verification for Deep Neural Networks](https://mlsanity.com/paper/2009.09663/) (2020)
- [CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors](https://mlsanity.com/paper/1903.01042/) (2019)
- [Efficient Soft-Error Detection for Low-precision Deep Learning Recommendation Models](https://mlsanity.com/paper/2103.00130/) (2021)
- [Enabling Software Resilience in GPGPU Applications via Partial Thread Protection](https://mlsanity.com/paper/2103.02825/) (2021)
- [FT-BLAS: A High Performance BLAS Implementation With Online Fault Tolerance](https://mlsanity.com/paper/2104.00897/) (2021)

## Built on by (most cited first, 9 of 9)

- [Understanding Silent Data Corruption in LLM Training](https://mlsanity.com/paper/2502.12340/) (2025)
- [FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions](https://mlsanity.com/paper/2112.13544/) (2021)
- [FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention](https://mlsanity.com/paper/2504.02211/) (2025)
- [Winograd Convolution: A Perspective from Fault Tolerance](https://mlsanity.com/paper/2202.08675/) (2022)
- [Resilience of Deep Learning applications: a systematic literature review of analysis and hardening techniques](https://mlsanity.com/paper/2309.16733/) (2023)
- [GCN-ABFT: Low-Cost Online Error Checking for Graph Convolutional Networks](https://mlsanity.com/paper/2412.18534/) (2024)
- [FT K-means: A High-Performance K-means on GPU with Fault Tolerance](https://mlsanity.com/paper/2408.01391/) (2024)
- [Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance](https://mlsanity.com/paper/2308.08230/) (2023)
- [Adaptive Soft Error Protection for Neural Network Processing](https://mlsanity.com/paper/2407.19664/) (2024)

## Abstract (arXiv)

Neural networks (NNs) are increasingly employed in safety-critical domains and in environments prone to unreliability (e.g., soft errors), such as on spacecraft. Therefore, it is critical to impart fault tolerance to NN inference. Algorithm-based fault tolerance (ABFT) is emerging as an efficient approach for fault tolerance in NNs. We propose an adaptive approach to ABFT for NN inference that exploits untapped opportunities in emerging deployment scenarios. GPUs have high compute-to-memory-bandwidth ratios, while NN layers have a wide range of arithmetic intensities. This leaves some layers compute bound and others memory-bandwidth bound, but current approaches to ABFT do not consider these differences. We first investigate ABFT schemes best suited for each of these scenarios. We then propose intensity-guided ABFT, an adaptive, arithmetic-intensity-guided approach that selects the most efficient ABFT scheme for each NN layer. Intensity-guided ABFT reduces execution-time overhead by 1.09--5.3× across many NNs compared to traditional approaches to ABFT.