# Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks

arXiv: https://arxiv.org/abs/1906.01017 · HTML: https://mlsanity.com/paper/1906.01017/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jun 2019.

## Built on by (most cited first, 30 of 57)

- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Bad Characters: Imperceptible NLP Attacks](https://mlsanity.com/paper/2106.09898/) (2021)
- ["Real Attackers Don't Compute Gradients": Bridging the Gap Between Adversarial ML Research and Practice](https://mlsanity.com/paper/2212.14315/) (2022)
- [Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits](https://mlsanity.com/paper/2102.10496/) (2021)
- [Qu-ANTI-zation: Exploiting Quantization Artifacts for Achieving Adversarial Outcomes](https://mlsanity.com/paper/2110.13541/) (2021)
- [RADAR: Run-time Adversarial Weight Attack Detection and Accuracy Recovery](https://mlsanity.com/paper/2101.08254/) (2021)
- [Towards Energy-Efficient and Secure Edge AI: A Cross-Layer Framework](https://mlsanity.com/paper/2109.09829/) (2021)
- [Versatile Weight Attack via Flipping Limited Bits](https://mlsanity.com/paper/2207.12405/) (2022)
- [FitAct: Error Resilient Deep Neural Networks via Fine-Grained Post-Trainable Activation Functions](https://mlsanity.com/paper/2112.13544/) (2021)
- [RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy](https://mlsanity.com/paper/2103.13813/) (2021)
- [DeepAxe: A Framework for Exploration of Approximation and Reliability Trade-offs in DNN Accelerators](https://mlsanity.com/paper/2303.08226/) (2023)
- [HASHTAG: Hash Signatures for Online Detection of Fault-Injection Attacks on Deep Neural Networks](https://mlsanity.com/paper/2111.01932/) (2021)
- [Towards a Safety Case for Hardware Fault Tolerance in Convolutional Neural Networks Using Activation Range Supervision](https://mlsanity.com/paper/2108.07019/) (2021)
- [Hardware faults that matter: Understanding and Estimating the safety impact of hardware faults on object detection DNNs](https://mlsanity.com/paper/2209.03225/) (2022)
- [Efficient Backdoor Attacks for Deep Neural Networks in Real-world Scenarios](https://mlsanity.com/paper/2306.08386/) (2023)
- [Data Free Backdoor Attacks](https://mlsanity.com/paper/2412.06219/) (2024)
- [Exploring Fault-Energy Trade-offs in Approximate DNN Hardware Accelerators](https://mlsanity.com/paper/2101.02860/) (2021)
- [FooBaR: Fault Fooling Backdoor Attack on Neural Network Training](https://mlsanity.com/paper/2109.11249/) (2021)
- [Don't Knock! Rowhammer at the Backdoor of DNN Models](https://mlsanity.com/paper/2110.07683/) (2021)
- [A Low-cost Strategic Monitoring Approach for Scalable and Interpretable Error Detection in Deep Neural Networks](https://mlsanity.com/paper/2310.20349/) (2023)
- [A Proxy Attack-Free Strategy for Practically Improving the Poisoning Efficiency in Backdoor Attacks](https://mlsanity.com/paper/2306.08313/) (2023)
- [PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips](https://mlsanity.com/paper/2412.07192/) (2024)
- [Maximal Brain Damage Without Data or Optimization: Disrupting Neural Networks via Sign-Bit Flips](https://mlsanity.com/paper/2502.07408/) (2025)
- [Compiled Models, Built-In Exploits: Uncovering Pervasive Bit-Flip Attack Surfaces in DNN Executables](https://mlsanity.com/paper/2309.06223/) (2023)
- [DeepNcode: Encoding-Based Protection against Bit-Flip Attacks on Neural Networks](https://mlsanity.com/paper/2405.13891/) (2024)
- [Designing DNNs for a trade-off between robustness and processing performance in embedded devices](https://mlsanity.com/paper/2412.03682/) (2024)
- [Thales: Formulating and Estimating Architectural Vulnerability Factors for DNN Accelerators](https://mlsanity.com/paper/2212.02649/) (2022)
- [DNN-Defender: A Victim-Focused In-DRAM Defense Mechanism for Taming Adversarial Weight Attack on DNNs](https://mlsanity.com/paper/2305.08034/) (2023)
- [Harden Deep Neural Networks Against Fault Injections Through Weight Scaling](https://mlsanity.com/paper/2411.18993/) (2024)
- [Verification of Bit-Flip Attacks against Quantized Neural Networks](https://mlsanity.com/paper/2502.16286/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) have been shown to tolerate "brain damage": cumulative changes to the network's parameters (e.g., pruning, numerical perturbations) typically result in a graceful degradation of classification accuracy. However, the limits of this natural resilience are not well understood in the presence of small adversarial changes to the DNN parameters' underlying memory representation, such as bit-flips that may be induced by hardware fault attacks. We study the effects of bitwise corruptions on 19 DNN models---six architectures on three image classification tasks---and we show that most models have at least one parameter that, after a specific bit-flip in their bitwise representation, causes an accuracy loss of over 90%. We employ simple heuristics to efficiently identify the parameters likely to be vulnerable. We estimate that 40-50% of the parameters in a model might lead to an accuracy drop greater than 10% when individually subjected to such single-bit perturbations. To demonstrate how an adversary could take advantage of this vulnerability, we study the impact of an exemplary hardware fault attack, Rowhammer, on DNNs. Specifically, we show that a Rowhammer enabled attacker co-located in the same physical machine can inflict significant accuracy drops (up to 99%) even with single bit-flip corruptions and no knowledge of the model. Our results expose the limits of DNNs' resilience against parameter perturbations induced by real-world fault attacks. We conclude by discussing possible mitigations and future research directions towards fault attack-resilient DNNs.