# Quasi-Newton's method in the class gradient defined high-curvature subspace

arXiv: https://arxiv.org/abs/2012.01938 · HTML: https://mlsanity.com/paper/2012.01938/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2020.

## Built on by (most cited first, 3 of 3)

- [Subspace Adversarial Training](https://mlsanity.com/paper/2111.12229/) (2021)
- [Low Dimensional Landscape Hypothesis is True: DNNs can be Trained in Tiny Subspaces](https://mlsanity.com/paper/2103.11154/) (2021)
- [Identifying Policy Gradient Subspaces](https://mlsanity.com/paper/2401.06604/) (2024)

## Abstract (arXiv)

Classification problems using deep learning have been shown to have a high-curvature subspace in the loss landscape equal in dimension to the number of classes. Moreover, this subspace corresponds to the subspace spanned by the logit gradients for each class. An obvious strategy to speed up optimisation would be to use Newton's method in the high-curvature subspace and stochastic gradient descent in the co-space. We show that a naive implementation actually slows down convergence and we speculate why this might be.