# FooBaR: Fault Fooling Backdoor Attack on Neural Network Training

arXiv: https://arxiv.org/abs/2109.11249 · HTML: https://mlsanity.com/paper/2109.11249/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 66 references that are in the map.
- First posted on arXiv 23 Sep 2021.

## Builds on (most cited first, 25 of 25)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain](https://mlsanity.com/paper/1708.06733/) (2017)
- [Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning](https://mlsanity.com/paper/1712.05526/) (2017)
- [Evasion Attacks against Machine Learning at Test Time](https://mlsanity.com/paper/1708.06131/) (2017)
- [Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks](https://mlsanity.com/paper/1805.12185/) (2018)
- [Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering](https://mlsanity.com/paper/1811.03728/) (2018)
- [Backdoor Learning: A Survey](https://mlsanity.com/paper/2007.08745/) (2020)
- [Wild Patterns: Ten Years After the Rise of Adversarial Machine Learning](https://mlsanity.com/paper/1712.03141/) (2017)
- [Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization](https://mlsanity.com/paper/1708.08689/) (2017)
- [Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation](https://mlsanity.com/paper/1808.10307/) (2018)
- [Backdoor Attacks to Graph Neural Networks](https://mlsanity.com/paper/2006.11165/) (2020)
- [BadEncoder: Backdoor Attacks to Pre-trained Encoders in Self-Supervised Learning](https://mlsanity.com/paper/2108.00352/) (2021)
- [TBT: Targeted Neural Network Attack with Bit Trojan](https://mlsanity.com/paper/1909.05193/) (2019)
- [Is feature selection secure against training data poisoning?](https://mlsanity.com/paper/1804.07933/) (2018)
- [A survey of deep neural network watermarking techniques](https://mlsanity.com/paper/2103.09274/) (2021)
- [An Analysis of Adversarial Attacks and Defenses on Autonomous Driving Models](https://mlsanity.com/paper/2002.02175/) (2020)
- [Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks](https://mlsanity.com/paper/1906.01017/) (2019)
- [Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits](https://mlsanity.com/paper/2102.10496/) (2021)
- [Data Poisoning Attacks against Online Learning](https://mlsanity.com/paper/1808.08994/) (2018)
- [Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networks](https://mlsanity.com/paper/1905.12032/) (2019)
- [SNIFF: Reverse Engineering of Neural Networks with Fault Attacks](https://mlsanity.com/paper/2002.11021/) (2020)
- [Rowhammer.js: A Remote Software-Induced Fault Attack in JavaScript](https://mlsanity.com/paper/1507.06955/) (2015)
- [Formal verification of a software countermeasure against instruction skip attacks](https://mlsanity.com/paper/1402.6461/) (2014)
- [V0LTpwn: Attacking x86 Processor Integrity from Software](https://mlsanity.com/paper/1912.04870/) (2019)

## Built on by (most cited first, 6 of 6)

- [TIJO: Trigger Inversion with Joint Optimization for Defending Multimodal Backdoored Models](https://mlsanity.com/paper/2308.03906/) (2023)
- [DeepNcode: Encoding-Based Protection against Bit-Flip Attacks on Neural Networks](https://mlsanity.com/paper/2405.13891/) (2024)
- [An Incremental Gray-box Physical Adversarial Attack on Neural Network Training](https://mlsanity.com/paper/2303.01245/) (2023)
- [DeepBaR: Fault Backdoor Attack on Deep Neural Network Layers](https://mlsanity.com/paper/2407.21220/) (2024)
- [The Weight of a Bit: EMFI Sensitivity Analysis of Embedded Deep Learning Models](https://mlsanity.com/paper/2602.16309/) (2026)
- [Capacity Overflow: A Blind Spot for Backdoor Attacks in Vision MoE](https://mlsanity.com/paper/2608.25371/) (2026)

## Abstract (arXiv)

Neural network implementations are known to be vulnerable to physical attack vectors such as fault injection attacks. As of now, these attacks were only utilized during the inference phase with the intention to cause a misclassification. In this work, we explore a novel attack paradigm by injecting faults during the training phase of a neural network in a way that the resulting network can be attacked during deployment without the necessity of further faulting. In particular, we discuss attacks against ReLU activation functions that make it possible to generate a family of malicious inputs, which are called fooling inputs, to be used at inference time to induce controlled misclassifications. Such malicious inputs are obtained by mathematically solving a system of linear equations that would cause a particular behaviour on the attacked activation functions, similar to the one induced in training through faulting. We call such attacks fooling backdoors as the fault attacks at the training phase inject backdoors into the network that allow an attacker to produce fooling inputs. We evaluate our approach against multi-layer perceptron networks and convolutional networks on a popular image classification task obtaining high attack success rates (from 60% to 100%) and high classification confidence when as little as 25 neurons are attacked while preserving high accuracy on the originally intended classification task.