mlsanityData to 7 Oct 2026

Neural Networks with Complex-Valued Weights Have No Spurious Local Minima

arXiv 2103.07287 ↗First posted 31 Jan 2021cs.LG stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

We study the benefits of complex-valued weights for neural networks. We prove that shallow complex neural networks with quadratic activations have no spurious local minima. In contrast, shallow real neural networks with quadratic activations have infinitely many spurious local minima under the same conditions. In addition, we provide specific examples to demonstrate that complex-valued weights turn poor local minima into saddle points.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 29 of its 48 references that are in the map. First posted on arXiv 31 Jan 2021.

Builds on

29 of 48 references.

References · 29

  1. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  2. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  3. 2015Unitary Evolution Recurrent Neural Networkscited by 291
  4. 2016Deep Learning without Poor Local Minimacited by 233
  5. 2017Deep Complex Networkscited by 215
  6. 2017Theoretical insights into the optimization landscape of over-parameterized shallow neural networkscited by 128
  7. 2016Identity Matters in Deep Learningcited by 107
  8. 2018On the Power of Over-parametrization in Neural Networks with Quadratic Activationcited by 102
  9. 2016Full-Capacity Unitary Recurrent Neural Networkscited by 90
  10. 2019Limitations of Lazy Training of Two-layers Neural Networkscited by 87
  11. 2017The loss surface of deep and wide neural networkscited by 74
  12. 2019Phase-aware Speech Enhancement with Deep Complex U-Netcited by 73
17 more
  1. 2017Deep linear neural networks with arbitrary loss: All local minima are globalcited by 64
  2. 2020The Global Landscape of Neural Networks: An Overviewcited by 57
  3. 2020Optimization and Generalization of Shallow Neural Networks with Quadratic Activation Functionscited by 50
  4. 2017Deep Quaternion Networkscited by 48
  5. 2018On the loss landscape of a class of deep neural networks with no bad local valleyscited by 38
  6. 2016Associative Long Short-Term Memorycited by 37
  7. 2017Depth Creates No Bad Local Minimacited by 29
  8. 2018Small nonlinearities in activation functions create bad local minima in neural networkscited by 28
  9. 2016On Complex Valued Convolutional Neural Networkscited by 26
  10. 2017Enhanced Radar Imaging Using a Complex-valued Convolutional Neural Networkcited by 18
  11. 2018Complex Gated Recurrent Neural Networkscited by 17
  12. 2020Network size and weights size for memorization with two-layers neural networkscited by 16
  13. 2018Complex Fully Convolutional Neural Networks for MR Image Reconstructioncited by 12
  14. 2019Stationary Points of Shallow Neural Networks with Quadratic Activation Functioncited by 12
  15. 2019Interpretable Complex-Valued Neural Networks for Privacy Protectioncited by 10
  16. 2019Depth creates no more spurious local minimacited by 5
  17. 2019Semi-supervised Complex-valued GAN for Polarimetric SAR Image Classificationcited by 4

Built on by

Not cited yet.