Hessian-aware Training for Enhancing DNNs Resilience to Parameter Corruptions
Not on the map. Why?
Abstract (arXiv)
Deep neural networks are not resilient to parameter corruptions: even a single-bitwise error in their parameters in memory can cause an accuracy drop of over 10%, and in the worst cases, up to 99%. This susceptibility poses great challenges in deploying models on computing platforms, where adversaries can induce bit-flips through software or bitwise corruptions may occur naturally. Most prior work addresses this issue with hardware or system-level approaches, such as integrating additional hardware components to verify a model's integrity at inference. However, these methods have not been widely deployed as they require infrastructure or platform-wide modifications. In this paper, we propose a new approach to addressing this issue: training models to be more resilient to bitwise corruptions to their parameters. Our approach, Hessian-aware training, promotes models with flatter loss surfaces. We show that, while there have been training methods, designed to improve generalization through Hessian-based approaches, they do not enhance resilience to parameter corruptions. In contrast, models trained with our method demonstrate increased resilience to parameter corruptions, particularly with a 20-50% reduction in the number of bits whose individual flipping leads to a 90-100% accuracy drop. Moreover, we show the synergy between ours and existing hardware and system-level defenses.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 48 references that are in the map. First posted on arXiv 2 Apr 2025.
Builds on
25 of 48 references.
References · 25
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
- 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
- 2020Sharpness-Aware Minimization for Efficiently Improving Generalizationcited by 1,417
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,352
- 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
- 2017Learning Efficient Convolutional Networks through Network Slimmingcited by 676
- 2017Exploring Generalization in Deep Learningcited by 600
- 2019Fantastic Generalization Measures and Where to Find Themcited by 513
- 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
- 2019PyHessian: Neural Networks Through the Lens of the Hessiancited by 292
13 more
- 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 261
- 2020ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learningcited by 202
- 2019TBT: Targeted Neural Network Attack with Bit Trojancited by 113
- 2019Bit-Flip Attack: Crushing Neural Network with Progressive Bit Searchcited by 108
- 2019Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attackscited by 57
- 2020DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flipscited by 56
- 2021DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memoriescited by 42
- 2021RADAR: Run-time Adversarial Weight Attack Detection and Accuracy Recoverycited by 18
- 2023Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networkscited by 17
- 2022Versatile Weight Attack via Flipping Limited Bitscited by 13
- 2021RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracycited by 11
- 2021HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performancecited by 8
- 2023DRAM-Locker: A General-Purpose DRAM Protection Mechanism against Adversarial DNN Weight Attackscited by 2
Built on by
Not cited yet.