# Newton methods based convolution neural networks using parallel processing

arXiv: https://arxiv.org/abs/2112.01401 · HTML: https://mlsanity.com/paper/2112.01401/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 56 references that are in the map.
- First posted on arXiv 2 Dec 2021.

## Builds on (most cited first, 9 of 9)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Practical Gauss-Newton Optimisation for Deep Learning](https://mlsanity.com/paper/1706.03662/) (2017)
- [Non-convex Optimization for Machine Learning](https://mlsanity.com/paper/1712.07897/) (2017)
- [Learning Internal Representations (COLT 1995)](https://mlsanity.com/paper/1911.05781/) (2019)
- [A Newton-CG Algorithm with Complexity Guarantees for Smooth Unconstrained Optimization](https://mlsanity.com/paper/1803.02924/) (2018)
- [Training Neural Networks with Stochastic Hessian-Free Optimization](https://mlsanity.com/paper/1301.3641/) (2013)
- [Distributed Newton Methods for Deep Neural Networks](https://mlsanity.com/paper/1802.00130/) (2018)
- [Newton Methods for Convolutional Neural Networks](https://mlsanity.com/paper/1811.06100/) (2018)
- [Distributed Hessian-Free Optimization for Deep Neural Network](https://mlsanity.com/paper/1606.00511/) (2016)

## Abstract (arXiv)

Training of convolutional neural networks is a high dimensional and a non-convex optimization problem. At present, it is inefficient in situations where parametric learning rates can not be confidently set. Some past works have introduced Newton methods for training deep neural networks. Newton methods for convolutional neural networks involve complicated operations. Finding the Hessian matrix in second-order methods becomes very complex as we mainly use the finite differences method with the image data. Newton methods for convolutional neural networks deals with this by using the sub-sampled Hessian Newton methods. In this paper, we have used the complete data instead of the sub-sampled methods that only handle partial data at a time. Further, we have used parallel processing instead of serial processing in mini-batch computations. The results obtained using parallel processing in this study, outperform the time taken by the previous approach.