# HERO: Hessian-Enhanced Robust Optimization for Unifying and Improving Generalization and Quantization Performance

arXiv: https://arxiv.org/abs/2111.11986 · HTML: https://mlsanity.com/paper/2111.11986/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 23 Nov 2021.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Sharpness-Aware Minimization for Efficiently Improving Generalization](https://mlsanity.com/paper/2010.01412/) (2020)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [AutoAugment: Learning Augmentation Policies from Data](https://mlsanity.com/paper/1805.09501/) (2018)
- [DivideMix: Learning with Noisy Labels as Semi-supervised Learning](https://mlsanity.com/paper/2002.07394/) (2020)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Model compression via distillation and quantization](https://mlsanity.com/paper/1802.05668/) (2018)
- [LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks](https://mlsanity.com/paper/1807.10029/) (2018)
- [Robustness via curvature regularization, and vice versa](https://mlsanity.com/paper/1811.09716/) (2018)
- [Improving Neural Network Quantization without Retraining using Outlier Channel Splitting](https://mlsanity.com/paper/1901.09504/) (2019)
- [BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization](https://mlsanity.com/paper/2102.10462/) (2021)

## Built on by (most cited first, 8 of 8)

- [Temperature Balancing, Layer-wise Weight Analysis, and Neural Network Training](https://mlsanity.com/paper/2312.00359/) (2023)
- [CLIBE: Detecting Dynamic Backdoors in Transformer-based NLP Models](https://mlsanity.com/paper/2409.01193/) (2024)
- [Taming Sensitive Weights : Noise Perturbation Fine-tuning for Robust LLM Quantization](https://mlsanity.com/paper/2412.06858/) (2024)
- [Titanus: Enabling KV Cache Pruning and Quantization On-the-Fly for LLM Acceleration](https://mlsanity.com/paper/2505.17787/) (2025)
- [Fisher-aware Quantization for DETR Detectors with Critical-category Objectives](https://mlsanity.com/paper/2407.03442/) (2024)
- [Quantization-Robust Unlearning through the Lens of Retain-Forget Loss Landscapes Interaction](https://mlsanity.com/paper/2609.27355/) (2026)
- [Hessian-aware Training for Enhancing DNNs Resilience to Parameter Corruptions](https://mlsanity.com/paper/2504.01933/) (2025)
- [When Flat Minima Fail: Characterizing INT4 Quantization Collapse After FP32 Convergence](https://mlsanity.com/paper/2604.15167/) (2026)

## Abstract (arXiv)

With the recent demand of deploying neural network models on mobile and edge devices, it is desired to improve the model's generalizability on unseen testing data, as well as enhance the model's robustness under fixed-point quantization for efficient deployment. Minimizing the training loss, however, provides few guarantees on the generalization and quantization performance. In this work, we fulfill the need of improving generalization and quantization performance simultaneously by theoretically unifying them under the framework of improving the model's robustness against bounded weight perturbation and minimizing the eigenvalues of the Hessian matrix with respect to model weights. We therefore propose HERO, a Hessian-enhanced robust optimization method, to minimize the Hessian eigenvalues through a gradient-based training process, simultaneously improving the generalization and quantization performance. HERO enables up to a 3.8% gain on test accuracy, up to 30% higher accuracy under 80% training label perturbation, and the best post-training quantization accuracy across a wide range of precision, including a >10% accuracy improvement over SGD-trained models for common model architectures on various datasets.