Regularization for convolutional kernel tensors to avoid unstable gradient problem in convolutional neural networks
Not on the map. Why?
Abstract (arXiv)
Convolutional neural networks are very popular nowadays. Training neural networks is not an easy task. Each convolution corresponds to a structured transformation matrix. In order to help avoid the exploding/vanishing gradient problem, it is desirable that the singular values of each transformation matrix are not large/small in the training process. We propose three new regularization terms for a convolutional kernel tensor to constrain the singular values of each transformation matrix. We show how to carry out the gradient type methods, which provides new insight about the training of convolutional neural networks.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 12 of its 21 references that are in the map. First posted on arXiv 5 Feb 2021.
Builds on
12 of 21 references.
References · 12
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2013Intriguing properties of neural networkscited by 5,114
- 2016Understanding deep learning requires rethinking generalizationcited by 1,657
- 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
- 2017Parseval Networks: Improving Robustness to Adversarial Examplescited by 288
- 2018Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networkscited by 193
- 2016A guide to convolution arithmetic for deep learningcited by 182
- 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
- 2019Orthogonal Convolutional Neural Networkscited by 122
- 2016Neural Photo Editing with Introspective Adversarial Networkscited by 109
- 2018The Singular Values of Convolutional Layerscited by 95
- 2019A Frobenius norm regularization method for convolutional kernels to avoid unstable gradient problemcited by 5
Built on by
Not cited yet.