# Input Hessian Regularization of Neural Networks

arXiv: https://arxiv.org/abs/2009.06571 · HTML: https://mlsanity.com/paper/2009.06571/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Sep 2020.

## Built on by (most cited first, 7 of 7)

- [A Unified Analysis of Mixed Sample Data Augmentation: A Loss Function Perspective](https://mlsanity.com/paper/2208.09913/) (2022)
- [Generalizing and Improving Jacobian and Hessian Regularization](https://mlsanity.com/paper/2212.00311/) (2022)
- [Selective Volume Mixup for Video Action Recognition](https://mlsanity.com/paper/2309.09534/) (2023)
- [From Robustness to Improved Generalization and Calibration in Pre-trained Language Models](https://mlsanity.com/paper/2404.00758/) (2024)
- [Training More Robust Classification Model via Discriminative Loss and Gaussian Noise Injection](https://mlsanity.com/paper/2405.18499/) (2024)
- [Robust Ranking Explanations](https://mlsanity.com/paper/2307.04024/) (2023)
- [Ensemble Adversarial Defense via Integration of Multiple Dispersed Low Curvature Models](https://mlsanity.com/paper/2403.16405/) (2024)

## Abstract (arXiv)

Regularizing the input gradient has shown to be effective in promoting the robustness of neural networks. The regularization of the input's Hessian is therefore a natural next step. A key challenge here is the computational complexity. Computing the Hessian of inputs is computationally infeasible. In this paper we propose an efficient algorithm to train deep neural networks with Hessian operator-norm regularization. We analyze the approach theoretically and prove that the Hessian operator norm relates to the ability of a neural network to withstand an adversarial attack. We give a preliminary experimental evaluation on the MNIST and FMNIST datasets, which demonstrates that the new regularizer can, indeed, be feasible and, furthermore, that it increases the robustness of neural networks over input gradient regularization.