# Hessian-aware Training for Enhancing DNNs Resilience to Parameter Corruptions

arXiv: https://arxiv.org/abs/2504.01933 · HTML: https://mlsanity.com/paper/2504.01933/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 48 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima](https://mlsanity.com/paper/1609.04836/) (2016)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [Fantastic Generalization Measures and Where to Find Them](https://mlsanity.com/paper/1912.02178/) (2019)
- [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019)
- [PyHessian: Neural Networks Through the Lens of the Hessian](https://mlsanity.com/paper/1912.07145/) (2019)
- [HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networks](https://mlsanity.com/paper/1911.03852/) (2019)
- [ADAHESSIAN: An Adaptive Second Order Optimizer for Machine Learning](https://mlsanity.com/paper/2006.00719/) (2020)
- [TBT: Targeted Neural Network Attack with Bit Trojan](https://mlsanity.com/paper/1909.05193/) (2019)
- [Bit-Flip Attack: Crushing Neural Network with Progressive Bit Search](https://mlsanity.com/paper/1903.12269/) (2019)
- [Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks](https://mlsanity.com/paper/1906.01017/) (2019)
- [DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips](https://mlsanity.com/paper/2003.13746/) (2020)
- [DeepSteal: Advanced Model Extractions Leveraging Efficient Weight Stealing in Memories](https://mlsanity.com/paper/2111.04625/) (2021)
- [RADAR: Run-time Adversarial Weight Attack Detection and Accuracy Recovery](https://mlsanity.com/paper/2101.08254/) (2021)
- [Aegis: Mitigating Targeted Bit-flip Attacks against Deep Neural Networks](https://mlsanity.com/paper/2302.13520/) (2023)
- [Versatile Weight Attack via Flipping Limited Bits](https://mlsanity.com/paper/2207.12405/) (2022)
- [RA-BNN: Constructing Robust & Accurate Binary Neural Network to Simultaneously Defend Adversarial Bit-Flip Attack and Improve Accuracy](https://mlsanity.com/paper/2103.13813/) (2021)
- [HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance](https://mlsanity.com/paper/2111.11986/) (2021)
- [DRAM-Locker: A General-Purpose DRAM Protection Mechanism against Adversarial DNN Weight Attacks](https://mlsanity.com/paper/2312.09027/) (2023)

## Abstract (arXiv)

Deep neural networks are not resilient to parameter corruptions: even a single-bitwise error in their parameters in memory can cause an accuracy drop of over 10%, and in the worst cases, up to 99%. This susceptibility poses great challenges in deploying models on computing platforms, where adversaries can induce bit-flips through software or bitwise corruptions may occur naturally. Most prior work addresses this issue with hardware or system-level approaches, such as integrating additional hardware components to verify a model's integrity at inference. However, these methods have not been widely deployed as they require infrastructure or platform-wide modifications. In this paper, we propose a new approach to addressing this issue: training models to be more resilient to bitwise corruptions to their parameters. Our approach, Hessian-aware training, promotes models with flatter loss surfaces. We show that, while there have been training methods, designed to improve generalization through Hessian-based approaches, they do not enhance resilience to parameter corruptions. In contrast, models trained with our method demonstrate increased resilience to parameter corruptions, particularly with a 20-50% reduction in the number of bits whose individual flipping leads to a 90-100% accuracy drop. Moreover, we show the synergy between ours and existing hardware and system-level defenses.