# First Power Linear Unit with Sign

arXiv: https://arxiv.org/abs/2111.14349 · HTML: https://mlsanity.com/paper/2111.14349/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 51 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 25 of 25)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Maxout Networks](https://mlsanity.com/paper/1302.4389/) (2013)
- [Searching for Activation Functions](https://mlsanity.com/paper/1710.05941/) (2017)
- [Mish: A Self Regularized Non-Monotonic Activation Function](https://mlsanity.com/paper/1908.08681/) (2019)
- [Understanding and Improving Convolutional Neural Networks via Concatenated Rectified Linear Units](https://mlsanity.com/paper/1603.05201/) (2016)
- [Continuously Differentiable Exponential Linear Units](https://mlsanity.com/paper/1704.07483/) (2017)
- [Dynamic ReLU](https://mlsanity.com/paper/2003.10027/) (2020)
- [Noisy Activation Functions](https://mlsanity.com/paper/1603.00391/) (2016)
- [Parametric Exponential Linear Unit for Deep Convolutional Neural Networks](https://mlsanity.com/paper/1605.09332/) (2016)
- [Activate or Not: Learning Customized Activation](https://mlsanity.com/paper/2009.04759/) (2020)
- [The Quest for the Golden Activation Function](https://mlsanity.com/paper/1808.00783/) (2018)
- [Funnel Activation for Visual Recognition](https://mlsanity.com/paper/2007.11824/) (2020)
- [Deep learning improved by biological activation functions](https://mlsanity.com/paper/1804.11237/) (2018)

## Abstract (arXiv)

This paper proposes a novel and insightful activation method termed FPLUS, which exploits mathematical power function with polar signs in form. It is enlightened by common inverse operation while endowed with an intuitive meaning of bionics. The formulation is derived theoretically under conditions of some prior knowledge and anticipative properties, and then its feasibility is verified through a series of experiments using typical benchmark datasets, whose results indicate our approach owns superior competitiveness among numerous activation functions, as well as compatible stability across many CNN architectures. Furthermore, we extend the function presented to a more generalized type called PFPLUS with two parameters that can be fixed or learnable, so as to augment its expressive capacity, and outcomes of identical tests validate this improvement.