# Neural Networks with Complex-Valued Weights Have No Spurious Local Minima

arXiv: https://arxiv.org/abs/2103.07287 · HTML: https://mlsanity.com/paper/2103.07287/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 48 references that are in the map.
- First posted on arXiv 31 Jan 2021.

## Builds on (most cited first, 29 of 29)

- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Deep Learning without Poor Local Minima](https://mlsanity.com/paper/1605.07110/) (2016)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Theoretical insights into the optimization landscape of over-parameterized shallow neural networks](https://mlsanity.com/paper/1707.04926/) (2017)
- [Identity Matters in Deep Learning](https://mlsanity.com/paper/1611.04231/) (2016)
- [On the Power of Over-parametrization in Neural Networks with Quadratic Activation](https://mlsanity.com/paper/1803.01206/) (2018)
- [Full-Capacity Unitary Recurrent Neural Networks](https://mlsanity.com/paper/1611.00035/) (2016)
- [Limitations of Lazy Training of Two-layers Neural Networks](https://mlsanity.com/paper/1906.08899/) (2019)
- [The loss surface of deep and wide neural networks](https://mlsanity.com/paper/1704.08045/) (2017)
- [Phase-aware Speech Enhancement with Deep Complex U-Net](https://mlsanity.com/paper/1903.03107/) (2019)
- [Deep linear neural networks with arbitrary loss: All local minima are global](https://mlsanity.com/paper/1712.01473/) (2017)
- [The Global Landscape of Neural Networks: An Overview](https://mlsanity.com/paper/2007.01429/) (2020)
- [Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functions](https://mlsanity.com/paper/2006.15459/) (2020)
- [Deep Quaternion Networks](https://mlsanity.com/paper/1712.04604/) (2017)
- [On the loss landscape of a class of deep neural networks with no bad local valleys](https://mlsanity.com/paper/1809.10749/) (2018)
- [Associative Long Short-Term Memory](https://mlsanity.com/paper/1602.03032/) (2016)
- [Depth Creates No Bad Local Minima](https://mlsanity.com/paper/1702.08580/) (2017)
- [Small nonlinearities in activation functions create bad local minima in neural networks](https://mlsanity.com/paper/1802.03487/) (2018)
- [On Complex Valued Convolutional Neural Networks](https://mlsanity.com/paper/1602.09046/) (2016)
- [Enhanced Radar Imaging Using a Complex-valued Convolutional Neural Network](https://mlsanity.com/paper/1712.10096/) (2017)
- [Complex Gated Recurrent Neural Networks](https://mlsanity.com/paper/1806.08267/) (2018)
- [Network size and weights size for memorization with two-layers neural networks](https://mlsanity.com/paper/2006.02855/) (2020)
- [Complex Fully Convolutional Neural Networks for MR Image Reconstruction](https://mlsanity.com/paper/1807.03343/) (2018)
- [Stationary Points of Shallow Neural Networks with Quadratic Activation Function](https://mlsanity.com/paper/1912.01599/) (2019)
- [Interpretable Complex-Valued Neural Networks for Privacy Protection](https://mlsanity.com/paper/1901.09546/) (2019)
- [Depth creates no more spurious local minima](https://mlsanity.com/paper/1901.09827/) (2019)
- [Semi-supervised Complex-valued GAN for Polarimetric SAR Image Classification](https://mlsanity.com/paper/1906.03605/) (2019)

## Abstract (arXiv)

We study the benefits of complex-valued weights for neural networks. We prove that shallow complex neural networks with quadratic activations have no spurious local minima. In contrast, shallow real neural networks with quadratic activations have infinitely many spurious local minima under the same conditions. In addition, we provide specific examples to demonstrate that complex-valued weights turn poor local minima into saddle points.