# Exploring Winograd Convolution for Cost-effective Neural Network Fault Tolerance

arXiv: https://arxiv.org/abs/2308.08230 · HTML: https://mlsanity.com/paper/2308.08230/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 66 references that are in the map.
- First posted on arXiv 16 Aug 2023.

## Builds on (most cited first, 22 of 22)

- [Fast Algorithms for Convolutional Neural Networks](https://mlsanity.com/paper/1509.09308/) (2015)
- [Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead](https://mlsanity.com/paper/2101.02559/) (2021)
- [TensorFI: A Flexible Fault Injection Framework for TensorFlow Applications](https://mlsanity.com/paper/2004.01743/) (2020)
- [FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks](https://mlsanity.com/paper/2003.12203/) (2020)
- [Efficient Sparse-Winograd Convolutional Neural Networks](https://mlsanity.com/paper/1802.06367/) (2018)
- [ThUnderVolt: Enabling Aggressive Voltage Underscaling and Timing Error Resilience for Energy Efficient Deep Neural Network Accelerators](https://mlsanity.com/paper/1802.03806/) (2018)
- [HarDNN: Feature Map Vulnerability Evaluation in CNNs](https://mlsanity.com/paper/2002.09786/) (2020)
- [FAT: Training Neural Networks for Reliable Inference Under Hardware Faults](https://mlsanity.com/paper/2011.05873/) (2020)
- [FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions](https://mlsanity.com/paper/2112.13544/) (2021)
- [Artificial Intelligence Safety and Cybersecurity: a Timeline of AI Failures](https://mlsanity.com/paper/1610.07997/) (2016)
- [Arithmetic-Intensity-Guided Fault Tolerance for Neural Network Inference on GPUs](https://mlsanity.com/paper/2104.09455/) (2021)
- [FTT-NAS: Discovering Fault-Tolerant Convolutional Neural Architecture](https://mlsanity.com/paper/2003.10375/) (2020)
- [Training Modern Deep Neural Networks for Memory-Fault Robustness](https://mlsanity.com/paper/1911.10287/) (2019)
- [R2F: A Remote Retraining Framework for AIoT Processors with Computing Errors](https://mlsanity.com/paper/2107.03096/) (2021)
- [Winograd Convolution: A Perspective from Fault Tolerance](https://mlsanity.com/paper/2202.08675/) (2022)
- [Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems](https://mlsanity.com/paper/2204.09514/) (2022)
- [DWM: A Decomposable Winograd Method for Convolution Acceleration](https://mlsanity.com/paper/2002.00552/) (2020)
- [Fault-Tolerant Deep Learning: A Hierarchical Perspective](https://mlsanity.com/paper/2204.01942/) (2022)
- [CodeNet: Training Large Scale Neural Networks in Presence of Soft-Errors](https://mlsanity.com/paper/1903.01042/) (2019)
- [LANCE: Efficient Low-Precision Quantized Winograd Convolution for Neural Networks Based on Graphics Processing Units](https://mlsanity.com/paper/2003.08646/) (2020)
- [Efficient Soft-Error Detection for Low-precision Deep Learning Recommendation Models](https://mlsanity.com/paper/2103.00130/) (2021)
- [Improving Energy Saving of One-sided Matrix Decompositions on CPU-GPU Heterogeneous Systems](https://mlsanity.com/paper/2301.03166/) (2023)

## Built on by (most cited first, 1 of 1)

- [Adaptive Soft Error Protection for Neural Network Processing](https://mlsanity.com/paper/2407.19664/) (2024)

## Abstract (arXiv)

Winograd is generally utilized to optimize convolution performance and computational efficiency because of the reduced multiplication operations, but the reliability issues brought by winograd are usually overlooked. In this work, we observe the great potential of winograd convolution in improving neural network (NN) fault tolerance. Based on the observation, we evaluate winograd convolution fault tolerance comprehensively from different granularities ranging from models, layers, and operation types for the first time. Then, we explore the use of inherent fault tolerance of winograd convolution for cost-effective NN protection against soft errors. Specifically, we mainly investigate how winograd convolution can be effectively incorporated with classical fault-tolerant design approaches including triple modular redundancy (TMR), fault-aware retraining, and constrained activation functions. According to our experiments, winograd convolution can reduce the fault-tolerant design overhead by 55.77% on average without any accuracy loss compared to standard convolution, and further reduce the computing overhead by 17.24% when the inherent fault tolerance of winograd convolution is considered. When it is applied on fault-tolerant neural networks enhanced with fault-aware retraining and constrained activation functions, the resulting model accuracy generally shows significant improvement in presence of various faults.