# Saddlepoints in Unsupervised Least Squares

arXiv: https://arxiv.org/abs/2104.05000 · HTML: https://mlsanity.com/paper/2104.05000/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 11 references that are in the map.
- First posted on arXiv 11 Apr 2021.

## Builds on (most cited first, 2 of 2)

- [Deep Learning without Poor Local Minima](https://mlsanity.com/paper/1605.07110/) (2016)
- [On the saddle point problem for non-convex optimization](https://mlsanity.com/paper/1405.4604/) (2014)

## Abstract (arXiv)

This paper sheds light on the risk landscape of unsupervised least squares in the context of deep auto-encoding neural nets. We formally establish an equivalence between unsupervised least squares and principal manifolds. This link provides insight into the risk landscape of auto--encoding under the mean squared error, in particular all non-trivial critical points are saddlepoints. Finding saddlepoints is in itself difficult, overcomplete auto-encoding poses the additional challenge that the saddlepoints are degenerate. Within this context we discuss regularization of auto-encoders, in particular bottleneck, denoising and contraction auto-encoding and propose a new optimization strategy that can be framed as particular form of contractive regularization.