# RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy

arXiv: https://arxiv.org/abs/2103.13813 · HTML: https://mlsanity.com/paper/2103.13813/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 47 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples](https://mlsanity.com/paper/1802.00420/) (2018)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithm](https://mlsanity.com/paper/1808.00278/) (2018)
- [Forward and Backward Information Retention for Accurate Binary Neural Networks](https://mlsanity.com/paper/1909.10788/) (2019)
- [Towards Accurate Binary Convolutional Neural Network](https://mlsanity.com/paper/1711.11294/) (2017)
- [TBT: Targeted Neural Network Attack with Bit Trojan](https://mlsanity.com/paper/1909.05193/) (2019)
- [Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search](https://mlsanity.com/paper/1903.12269/) (2019)
- [DARTS+: Improved Differentiable Architecture Search with Early Stopping](https://mlsanity.com/paper/1909.06035/) (2019)
- [XNOR-Net++: Improved Binary Neural Networks](https://mlsanity.com/paper/1909.13863/) (2019)
- [Rotated Binary Neural Network](https://mlsanity.com/paper/2009.13055/) (2020)
- [Adversarial Laser Beam: Effective Physical-World Attack to DNNs in a Blink](https://mlsanity.com/paper/2103.06504/) (2021)
- [Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks](https://mlsanity.com/paper/1906.01017/) (2019)
- [DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips](https://mlsanity.com/paper/2003.13746/) (2020)
- [WRPN: Wide Reduced-Precision Networks](https://mlsanity.com/paper/1709.01134/) (2017)
- [Targeted Attack against Deep Neural Networks via Flipping Limited Weight Bits](https://mlsanity.com/paper/2102.10496/) (2021)
- [XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inference](https://mlsanity.com/paper/1807.03010/) (2018)
- [Growing Efficient Deep Networks by Structured Continuous Sparsification](https://mlsanity.com/paper/2007.15353/) (2020)
- [Fault Sneaking Attack: a Stealthy Framework for Misleading Deep Neural Networks](https://mlsanity.com/paper/1905.12032/) (2019)
- [FT-CNN: Algorithm-Based Fault Tolerance for Convolutional Neural Networks](https://mlsanity.com/paper/2003.12203/) (2020)
- [RADAR: Run-time Adversarial Weight Attack Detection and Accuracy Recovery](https://mlsanity.com/paper/2101.08254/) (2021)
- [Regularized Binary Network Training](https://mlsanity.com/paper/1812.11800/) (2018)
- [Another Flip in the Wall of Rowhammer Defenses](https://mlsanity.com/paper/1710.00551/) (2017)
- [DeepDyve: Dynamic Verification for Deep Neural Networks](https://mlsanity.com/paper/2009.09663/) (2020)
- [Functional Error Correction for Robust Neural Networks](https://mlsanity.com/paper/2001.03814/) (2020)

## Built on by (most cited first, 11 of 11)

- [HASHTAG: Hash Signatures for Online Detection of Fault-Injection Attacks on Deep Neural Networks](https://mlsanity.com/paper/2111.01932/) (2021)
- [PrisonBreak: Jailbreaking Large Language Models with at Most Twenty-Five Targeted Bit-flips](https://mlsanity.com/paper/2412.07192/) (2024)
- [Maximal Brain Damage Without Data or Optimization: Disrupting Neural Networks via Sign-Bit Flips](https://mlsanity.com/paper/2502.07408/) (2025)
- [DeepNcode: Encoding-Based Protection against Bit-Flip Attacks on Neural Networks](https://mlsanity.com/paper/2405.13891/) (2024)
- [DNN-Defender: A Victim-Focused In-DRAM Defense Mechanism for Taming Adversarial Weight Attack on DNNs](https://mlsanity.com/paper/2305.08034/) (2023)
- [DeepBaR: Fault Backdoor Attack on Deep Neural Network Layers](https://mlsanity.com/paper/2407.21220/) (2024)
- [Targeted Bit-Flip Attacks on LLM-Based Agents](https://mlsanity.com/paper/2603.10042/) (2026)
- [Bit-Flip Attacks on Vision-Language-Action Models: Action-Decoding Architecture Shapes the Vulnerability](https://mlsanity.com/paper/2608.15475/) (2026)
- [An Empirical Study of Aegis](https://mlsanity.com/paper/2404.15784/) (2024)
- [Hessian-aware Training for Enhancing DNNs Resilience to Parameter Corruptions](https://mlsanity.com/paper/2504.01933/) (2025)
- [Quantum Tunneling-Aware Machine Learning: Physics-Derived Noise Models for Robust Deployment](https://mlsanity.com/paper/2606.00741/) (2026)

## Abstract (arXiv)

Recently developed adversarial weight attack, a.k.a. bit-flip attack (BFA), has shown enormous success in compromising Deep Neural Network (DNN) performance with an extremely small amount of model parameter perturbation. To defend against this threat, we propose RA-BNN that adopts a complete binary (i.e., for both weights and activation) neural network (BNN) to significantly improve DNN model robustness (defined as the number of bit-flips required to degrade the accuracy to as low as a random guess). However, such an aggressive low bit-width model suffers from poor clean (i.e., no attack) inference accuracy. To counter this, we propose a novel and efficient two-stage network growing method, named Early-Growth. It selectively grows the channel size of each BNN layer based on channel-wise binary masks training with Gumbel-Sigmoid function. Apart from recovering the inference accuracy, our RA-BNN after growing also shows significantly higher resistance to BFA. Our evaluation of the CIFAR-10 dataset shows that the proposed RA-BNN can improve the clean model accuracy by ~2-8 %, compared with a baseline BNN, while simultaneously improving the resistance to BFA by more than 125 x. Moreover, on ImageNet, with a sufficiently large (e.g., 5,000) amount of bit-flips, the baseline BNN accuracy drops to 4.3 % from 51.9 %, while our RA-BNN accuracy only drops to 37.1 % from 60.9 % (9 % clean accuracy improvement).