# A Kernel Perspective for Regularizing Deep Neural Networks

arXiv: https://arxiv.org/abs/1810.00363 · HTML: https://mlsanity.com/paper/1810.00363/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Sep 2018.

## Built on by (most cited first, 4 of 4)

- [Unrolled Variational Bayesian Algorithm for Image Blind Deconvolution](https://mlsanity.com/paper/2110.07202/) (2021)
- [Jacobian Norm with Selective Input Gradient Regularization for Improved and Interpretable Adversarial Defense](https://mlsanity.com/paper/2207.13036/) (2022)
- [A Fast and Efficient Conditional Learning for Tunable Trade-Off between Accuracy and Robustness](https://mlsanity.com/paper/2204.00426/) (2022)
- [On Inductive Biases for Machine Learning in Data Constrained Settings](https://mlsanity.com/paper/2302.10692/) (2023)

## Abstract (arXiv)

We propose a new point of view for regularizing deep neural networks by using the norm of a reproducing kernel Hilbert space (RKHS). Even though this norm cannot be computed, it admits upper and lower approximations leading to various practical strategies. Specifically, this perspective (i) provides a common umbrella for many existing regularization principles, including spectral norm and gradient penalties, or adversarial training, (ii) leads to new effective regularization penalties, and (iii) suggests hybrid strategies combining lower and upper bounds to get better approximations of the RKHS norm. We experimentally show this approach to be effective when learning on small datasets, or to obtain adversarially robust models.