# Towards Understanding Theoretical Advantages of Complex-Reaction Networks

arXiv: https://arxiv.org/abs/2108.06711 · HTML: https://mlsanity.com/paper/2108.06711/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 49 references that are in the map.
- First posted on arXiv 15 Aug 2021.

## Builds on (most cited first, 24 of 24)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Complex Embeddings for Simple Link Prediction](https://mlsanity.com/paper/1606.06357/) (2016)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Speech Recognition with Deep Recurrent Neural Networks](https://mlsanity.com/paper/1303.5778/) (2013)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [Fine-Grained Analysis of Optimization and Generalization for Overparameterized Two-Layer Neural Networks](https://mlsanity.com/paper/1901.08584/) (2019)
- [Train faster, generalize better: Stability of stochastic gradient descent](https://mlsanity.com/paper/1509.01240/) (2015)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Harmonic Networks: Deep Translation and Rotation Equivariance](https://mlsanity.com/paper/1612.04642/) (2016)
- [Identifying and attacking the saddle point problem in high-dimensional non-convex optimization](https://mlsanity.com/paper/1406.2572/) (2014)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [Universal Approximation with Deep Narrow Networks](https://mlsanity.com/paper/1905.08539/) (2019)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization](https://mlsanity.com/paper/1908.09375/) (2019)
- [Deep Roto-Translation Scattering for Object Classification](https://mlsanity.com/paper/1412.8659/) (2014)
- [Associative Long Short-Term Memory](https://mlsanity.com/paper/1602.03032/) (2016)
- [Better than Real: Complex-valued Neural Nets for MRI Fingerprinting](https://mlsanity.com/paper/1707.00070/) (2017)
- [The universal approximation theorem for complex-valued neural networks](https://mlsanity.com/paper/2012.03351/) (2020)
- [Complex Gated Recurrent Neural Networks](https://mlsanity.com/paper/1806.08267/) (2018)
- [A mathematical motivation for complex-valued convolutional networks](https://mlsanity.com/paper/1503.03438/) (2015)
- [Flexible Transmitter Network](https://mlsanity.com/paper/2004.03839/) (2020)

## Built on by (most cited first, 6 of 6)

- [Towards NeuroAI: Introducing Neuronal Diversity into Artificial Neural Networks](https://mlsanity.com/paper/2301.09245/) (2023)
- [Theoretical Exploration of Flexible Transmitter Model](https://mlsanity.com/paper/2111.06027/) (2021)
- [LiteFusion: Taming 3D Object Detectors from Vision-Based to Multi-Modal with Minimal Adaptation](https://mlsanity.com/paper/2512.20217/) (2025)
- [On the Approximation and Complexity of Deep Neural Networks to Invariant Functions](https://mlsanity.com/paper/2210.15279/) (2022)
- [On the Computational Complexities of Complex-valued Neural Networks](https://mlsanity.com/paper/2310.13075/) (2023)
- [The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest](https://mlsanity.com/paper/2407.05108/) (2024)

## Abstract (arXiv)

Complex-valued neural networks have attracted increasing attention in recent years, while it remains open on the advantages of complex-valued neural networks in comparison with real-valued networks. This work takes one step on this direction by introducing the complex-reaction network with fully-connected feed-forward architecture. We prove the universal approximation property for complex-reaction networks, and show that a class of radial functions can be approximated by a complex-reaction network using the polynomial number of parameters, whereas real-valued networks need at least exponential parameters to reach the same approximation level. For empirical risk minimization, our theoretical result shows that the critical point set of complex-reaction networks is a proper subset of that of real-valued networks, which may show some insights on finding the optimal solutions more easily for complex-reaction networks.