# Squareplus: A Softplus-Like Algebraic Rectifier

arXiv: https://arxiv.org/abs/2112.11687 · HTML: https://mlsanity.com/paper/2112.11687/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 10 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Builds on (most cited first, 3 of 3)

- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [A General and Adaptive Robust Loss Function](https://mlsanity.com/paper/1701.03077/) (2017)

## Built on by (most cited first, 19 of 19)

- [NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fields](https://mlsanity.com/paper/2210.04932/) (2022)
- [L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learning](https://mlsanity.com/paper/2202.07152/) (2022)
- [Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylization](https://mlsanity.com/paper/2303.13232/) (2023)
- [Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizes](https://mlsanity.com/paper/2406.05033/) (2024)
- [Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Search](https://mlsanity.com/paper/2212.10765/) (2022)
- [Soft Actor-Critic Algorithm with Truly-satisfied Inequality Constraint](https://mlsanity.com/paper/2303.04356/) (2023)
- [Revisiting Experience Replayable Conditions](https://mlsanity.com/paper/2402.10374/) (2024)
- [Large Deviations of Gaussian Neural Networks with ReLU activation](https://mlsanity.com/paper/2405.16958/) (2024)
- [LiRA: Light-Robust Adversary for Model-based Reinforcement Learning in Real World](https://mlsanity.com/paper/2409.19617/) (2024)
- [Flexible Empowerment at Reasoning with Extended Best-of-N Sampling](https://mlsanity.com/paper/2604.15614/) (2026)
- [Design of Restricted Normalizing Flow towards Arbitrary Stochastic Policy with Computational Efficiency](https://mlsanity.com/paper/2412.12894/) (2024)
- [Neural Cone Radiosity for Interactive Global Illumination with Glossy Materials](https://mlsanity.com/paper/2509.07522/) (2025)
- [DL101 Neural Network Outputs and Loss Functions](https://mlsanity.com/paper/2511.05131/) (2025)
- [Pseudo-Quantized Actor-Critic Algorithm for Robustness to Noisy Temporal Difference Error](https://mlsanity.com/paper/2604.01613/) (2026)
- [sLTN: Structural Logic Tensor Networks](https://mlsanity.com/paper/2608.11136/) (2026)
- [Gradient Networks for Universal Magnetic Modeling of Synchronous Machines](https://mlsanity.com/paper/2602.14947/) (2026)
- [IGLU: The Integrated Gaussian Linear Unit Activation Function](https://mlsanity.com/paper/2603.06861/) (2026)
- [Implementation of Hyperelastic Physics-Augmented Neural Networks in the Explicit Finite Element Codes Simcenter Radioss and OpenRadioss with Applications to Impact Events](https://mlsanity.com/paper/2606.29874/) (2026)
- [The Spectral Neuron](https://mlsanity.com/paper/2608.08003/) (2026)

## Abstract (arXiv)

We present squareplus, an activation function that resembles softplus, but which can be computed using only algebraic operations: addition, multiplication, and square-root. Because squareplus is ~6x faster to evaluate than softplus on a CPU and does not require access to transcendental functions, it may have practical value in resource-limited deep learning applications.