# Theoretical Exploration of Flexible Transmitter Model

arXiv: https://arxiv.org/abs/2111.06027 · HTML: https://mlsanity.com/paper/2111.06027/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 69 references that are in the map.
- First posted on arXiv 11 Nov 2021.

## Builds on (most cited first, 30 of 30)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [A Convergence Theory for Deep Learning via Over-Parameterization](https://mlsanity.com/paper/1811.03962/) (2018)
- [Gradient Descent Provably Optimizes Over-parameterized Neural Networks](https://mlsanity.com/paper/1810.02054/) (2018)
- [Gradient Descent Finds Global Minima of Deep Neural Networks](https://mlsanity.com/paper/1811.03804/) (2018)
- [The Expressive Power of Neural Networks: A View from the Width](https://mlsanity.com/paper/1709.02540/) (2017)
- [Unitary Evolution Recurrent Neural Networks](https://mlsanity.com/paper/1511.06464/) (2015)
- [Learning Overparameterized Neural Networks via Stochastic Gradient Descent on Structured Data](https://mlsanity.com/paper/1808.01204/) (2018)
- [Benefits of depth in neural networks](https://mlsanity.com/paper/1602.04485/) (2016)
- [Deep Learning without Poor Local Minima](https://mlsanity.com/paper/1605.07110/) (2016)
- [Loss landscapes and optimization in over-parameterized non-linear systems and neural networks](https://mlsanity.com/paper/2003.00307/) (2020)
- [Deep Complex Networks](https://mlsanity.com/paper/1705.09792/) (2017)
- [The Power of Depth for Feedforward Neural Networks](https://mlsanity.com/paper/1512.03965/) (2015)
- [Universality of Deep Convolutional Neural Networks](https://mlsanity.com/paper/1805.10769/) (2018)
- [An Improved Analysis of Training Over-parameterized Deep Neural Networks](https://mlsanity.com/paper/1906.04688/) (2019)
- [On the Convergence Rate of Training Recurrent Neural Networks](https://mlsanity.com/paper/1810.12065/) (2018)
- [The loss surface of deep and wide neural networks](https://mlsanity.com/paper/1704.08045/) (2017)
- [Deep linear neural networks with arbitrary loss: All local minima are global](https://mlsanity.com/paper/1712.01473/) (2017)
- [Depth-Width Tradeoffs in Approximating Natural Functions with Neural Networks](https://mlsanity.com/paper/1610.09887/) (2016)
- [Learning ReLUs via Gradient Descent](https://mlsanity.com/paper/1705.04591/) (2017)
- [On the loss landscape of a class of deep neural networks with no bad local valleys](https://mlsanity.com/paper/1809.10749/) (2018)
- [The universal approximation theorem for complex-valued neural networks](https://mlsanity.com/paper/2012.03351/) (2020)
- [Understanding the Loss Surface of Neural Networks for Binary Classification](https://mlsanity.com/paper/1803.00909/) (2018)
- [On Complex Valued Convolutional Neural Networks](https://mlsanity.com/paper/1602.09046/) (2016)
- [Adding One Neuron Can Eliminate All Bad Local Minima](https://mlsanity.com/paper/1805.08671/) (2018)
- [Depth with Nonlinearity Creates No Bad Local Minima in ResNets](https://mlsanity.com/paper/1810.09038/) (2018)
- [Elimination of All Bad Local Minima in Deep Learning](https://mlsanity.com/paper/1901.00279/) (2019)
- [Towards Understanding Theoretical Advantages of Complex-Reaction Networks](https://mlsanity.com/paper/2108.06711/) (2021)
- [Bifurcation Spiking Neural Network](https://mlsanity.com/paper/1909.08341/) (2019)
- [Flexible Transmitter Network](https://mlsanity.com/paper/2004.03839/) (2020)

## Built on by (most cited first, 3 of 3)

- [Bridging Today and the Future of Humanity: AI Safety in 2024 and Beyond](https://mlsanity.com/paper/2410.18114/) (2024)
- [On the Approximation and Complexity of Deep Neural Networks to Invariant Functions](https://mlsanity.com/paper/2210.15279/) (2022)
- [The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest](https://mlsanity.com/paper/2407.05108/) (2024)

## Abstract (arXiv)

Neural network models generally involve two important components, i.e., network architecture and neuron model. Although there are abundant studies about network architectures, only a few neuron models have been developed, such as the MP neuron model developed in 1943 and the spiking neuron model developed in the 1950s. Recently, a new bio-plausible neuron model, Flexible Transmitter (FT) model, has been proposed. It exhibits promising behaviors, particularly on temporal-spatial signals, even when simply embedded into the common feedforward network architecture. This paper attempts to understand the properties of the FT network (FTNet) theoretically. Under mild assumptions, we show that: i) FTNet is a universal approximator; ii) the approximation complexity of FTNet can be exponentially smaller than those of commonly-used real-valued neural networks with feedforward/recurrent architectures and is of the same order in the worst case; iii) any local minimum of FTNet is the global minimum, implying that it is possible to identify global minima by local search algorithms.