# A Frobenius norm regularization method for convolutional kernels to avoid unstable gradient problem

arXiv: https://arxiv.org/abs/1907.11235 · HTML: https://mlsanity.com/paper/1907.11235/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Jul 2019.

## Built on by (most cited first, 5 of 5)

- [Geometry and Generalization: Eigenvalues as predictors of where a network will fail to generalize](https://mlsanity.com/paper/2107.06386/) (2021)
- [GCI-ViTAL: Gradual Confidence Improvement with Vision Transformers for Active Learning on Label Noise](https://mlsanity.com/paper/2411.05939/) (2024)
- [Non-Negative Stiefel Approximating Flow: Orthogonalish Matrix Optimization for Interpretable Embeddings](https://mlsanity.com/paper/2511.06425/) (2025)
- [Regularization for convolutional kernel tensors to avoid unstable gradient problem in convolutional neural networks](https://mlsanity.com/paper/2102.04294/) (2021)
- [Spectral Complexity-scaled Generalization Bound of Complex-valued Neural Networks](https://mlsanity.com/paper/2112.03467/) (2021)

## Abstract (arXiv)

Convolutional neural network is a very important model of deep learning. It can help avoid the exploding/vanishing gradient problem and improve the generalizability of a neural network if the singular values of the Jacobian of a layer are bounded around 1 in the training process. We propose a new penalty function for a convolutional kernel to let the singular values of the corresponding transformation matrix are bounded around 1. We show how to carry out the gradient type methods. The penalty is about the structured transformation matrix corresponding to a convolutional kernel. This provides a new regularization method about the weights of convolutional layers.