# Soft Error Reliability Analysis of Vision Transformers

arXiv: https://arxiv.org/abs/2302.10468 · HTML: https://mlsanity.com/paper/2302.10468/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Feb 2023.

## Built on by (most cited first, 10 of 10)

- [FT-Transformer: Resilient and Reliable Transformer with End-to-End Fault Tolerant Attention](https://mlsanity.com/paper/2504.02211/) (2025)
- [MRFI: An Open Source Multi-Resolution Fault Injection Framework for Neural Network Processing](https://mlsanity.com/paper/2306.11758/) (2023)
- [Effective and Memory-Efficient Alternatives to ECC for Reliable Large-Scale DNNs](https://mlsanity.com/paper/2605.07417/) (2026)
- [Global Clipper: Enhancing Safety and Reliability of Transformer-based Object Detection Models](https://mlsanity.com/paper/2406.03229/) (2024)
- [Zero Memory Overhead Approach for Protecting Vision Transformer Parameters](https://mlsanity.com/paper/2507.03816/) (2025)
- [SENTRY: Statistical Reliability Analysis of Vision Transformers Under Soft Errors](https://mlsanity.com/paper/2606.07620/) (2026)
- [Concurrent Linguistic Error Detection (CLED): a New Methodology for Error Detection in Large Language Models](https://mlsanity.com/paper/2403.16393/) (2024)
- [Adaptive Soft Error Protection for Neural Network Processing](https://mlsanity.com/paper/2407.19664/) (2024)
- [Analysis of LLM Vulnerability to GPU Soft Errors: An Instruction-Level Fault Injection Study](https://mlsanity.com/paper/2601.19912/) (2025)
- [WARD: Runtime Workload-Adaptive Vision TRansformer Framework for Dependable Edge AI](https://mlsanity.com/paper/2609.17556/) (2026)

## Abstract (arXiv)

Vision Transformers (ViTs) that leverage self-attention mechanism have shown superior performance on many classical vision tasks compared to convolutional neural networks (CNNs) and gain increasing popularity recently. Existing ViTs works mainly optimize performance and accuracy, but ViTs reliability issues induced by soft errors in large-scale VLSI designs have generally been overlooked. In this work, we mainly study the reliability of ViTs and investigate the vulnerability from different architecture granularities ranging from models, layers, modules, and patches for the first time. The investigation reveals that ViTs with the self-attention mechanism are generally more resilient on linear computing including general matrix-matrix multiplication (GEMM) and full connection (FC) and show a relatively even vulnerability distribution across the patches. ViTs involve more fragile non-linear computing such as softmax and GELU compared to typical CNNs. With the above observations, we propose a lightweight block-wise algorithm-based fault tolerance (LB-ABFT) approach to protect the linear computing implemented with distinct sizes of GEMM and apply a range-based protection scheme to mitigate soft errors in non-linear computing. According to our experiments, the proposed fault-tolerant approaches enhance ViTs accuracy significantly with minor computing overhead in presence of various soft errors.