mlsanityData to 7 Oct 2026

Towards Understanding Theoretical Advantages of Complex-Reaction Networks

arXiv 2108.06711 ↗First posted 15 Aug 2021cs.LG cs.AIcited by 6

Not on the map. Why?

Abstract (arXiv)

Complex-valued neural networks have attracted increasing attention in recent years, while it remains open on the advantages of complex-valued neural networks in comparison with real-valued networks. This work takes one step on this direction by introducing the complex-reaction network with fully-connected feed-forward architecture. We prove the universal approximation property for complex-reaction networks, and show that a class of radial functions can be approximated by a complex-reaction network using the polynomial number of parameters, whereas real-valued networks need at least exponential parameters to reach the same approximation level. For empirical risk minimization, our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks, which may show some insights on finding the optimal solutions more easily for complex-reaction networks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 24 of its 49 references that are in the map. First posted on arXiv 15 Aug 2021.

Builds on

24 of 49 references.

References · 24

  1. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  2. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  3. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
  4. 2016Complex Embeddings for Simple Link Predictioncited by 897
  5. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  6. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 666
  7. 2018Gradient Descent Provably Optimizes Over-parameterized Neural Networkscited by 655
  8. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  9. 2019Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networkscited by 562
  10. 2015Train faster, generalize better: Stability of stochastic gradient descentcited by 547
  11. 2017The Expressive Power of Neural Networks: A View from the Widthcited by 318
  12. 2016Harmonic Networks: Deep Translation and Rotation Equivariancecited by 293
12 more
  1. 2014Identifying and attacking the saddle point problem in high-dimensional non-convex optimizationcited by 286
  2. 2017Deep Complex Networkscited by 215
  3. 2019Universal Approximation with Deep Narrow Networkscited by 198
  4. 2015The Power of Depth for Feedforward Neural Networkscited by 181
  5. 2019Theoretical Issues in Deep Networks: Approximation, Optimization and Generalizationcited by 72
  6. 2014Deep Roto-Translation Scattering for Object Classificationcited by 49
  7. 2016Associative Long Short-Term Memorycited by 37
  8. 2017Better than Real: Complex-valued Neural Nets for MRI Fingerprintingcited by 30
  9. 2020The universal approximation theorem for complex-valued neural networkscited by 28
  10. 2018Complex Gated Recurrent Neural Networkscited by 17
  11. 2015A mathematical motivation for complex-valued convolutional networkscited by 13
  12. 2020Flexible Transmitter Networkcited by 4

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 1; 2023: 2; 2024: 1; 2025: 1.

6 papers.

  1. 2023Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networkscited by 6
  2. 2021Theoretical Exploration of Flexible Transmitter Modelcited by 3
  3. 2025LiteFusion: Taming 3D Object Detectors from Vision-Based to Multi-Modal with Minimal Adaptationcited by 1
  4. 2022On the Approximation and Complexity of Deep Neural Networks to Invariant Functionscited by 0
  5. 2023On the Computational Complexities of Complex-valued Neural Networkscited by 0
  6. 2024The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forestcited by 0