# A New Artificial Neuron Proposal with Trainable Simultaneous Local and Global Activation Function

arXiv: https://arxiv.org/abs/2101.06100 · HTML: https://mlsanity.com/paper/2101.06100/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 4 of its 26 references that are in the map.
- First posted on arXiv 15 Jan 2021.

## Builds on (most cited first, 4 of 4)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Unsupervised Neural Networks for Quantum Eigenvalue Problems](https://mlsanity.com/paper/2010.05075/) (2020)
- [Semi-supervised Neural Networks solve an inverse problem for modeling Covid-19 spread](https://mlsanity.com/paper/2010.05074/) (2020)
- [Unsupervised Learning of Solutions to Differential Equations with Generative Adversarial Networks](https://mlsanity.com/paper/2007.11133/) (2020)

## Abstract (arXiv)

The activation function plays a fundamental role in the artificial neural network learning process. However, there is no obvious choice or procedure to determine the best activation function, which depends on the problem. This study proposes a new artificial neuron, named global-local neuron, with a trainable activation function composed of two components, a global and a local. The global component term used here is relative to a mathematical function to describe a general feature present in all problem domain. The local component is a function that can represent a localized behavior, like a transient or a perturbation. This new neuron can define the importance of each activation function component in the learning phase. Depending on the problem, it results in a purely global, or purely local, or a mixed global and local activation function after the training phase. Here, the trigonometric sine function was employed for the global component and the hyperbolic tangent for the local component. The proposed neuron was tested for problems where the target was a purely global function, or purely local function, or a composition of two global and local functions. Two classes of test problems were investigated, regression problems and differential equations solving. The experimental tests demonstrated the Global-Local Neuron network's superior performance, compared with simple neural networks with sine or hyperbolic tangent activation function, and with a hybrid network that combines these two simple neural networks.