mlsanityData to 14 Oct 2026

Squareplus: A Softplus-Like Algebraic Rectifier

arXiv 2112.11687 ↗First posted 22 Dec 2021cs.LG cs.NEcited by 19

Not on the map. Why?

Abstract (arXiv)

We present squareplus, an activation function that resembles softplus, but which can be computed using only algebraic operations: addition, multiplication, and square-root. Because squareplus is ~6x faster to evaluate than softplus on a CPU and does not require access to transcendental functions, it may have practical value in resource-limited deep learning applications.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 3 of its 10 references that are in the map. First posted on arXiv 22 Dec 2021.

Builds on

3 of 10 references.

References · 3

  1. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,070
  2. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 950
  3. 2017A General and Adaptive Robust Loss Functioncited by 236

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 2; 2024: 5; 2025: 2; 2026: 7.

19 papers.

Sort:

By area:

  1. 2022NeRF2Real: Sim2real Transfer of Vision-guided Bipedal Motion Skills using Neural Radiance Fieldscited by 44
  2. 2022L2C2: Locally Lipschitz Continuous Constraint towards Stable and Smooth Reinforcement Learningcited by 27
  3. 2023Transforming Radiance Field with Lipschitz Network for Photorealistic 3D Scene Stylizationcited by 14
  4. 2024Gradient Descent on Logistic Regression with Non-Separable Data and Large Step Sizescited by 14
  5. 2022Reward Bonuses with Gain Scheduling Inspired by Iterative Deepening Searchcited by 5
  6. 2023Soft Actor-Critic Algorithm with Truly-satisfied Inequality Constraintcited by 3
  7. 2024Revisiting Experience Replayable Conditionscited by 3
  8. 2024Large Deviations of Gaussian Neural Networks with ReLU activationcited by 3
  9. 2024LiRA: Light-Robust Adversary for Model-based Reinforcement Learning in Real Worldcited by 3
  10. 2026Flexible Empowerment at Reasoning with Extended Best-of-N Samplingcited by 3
  11. 2024Design of Restricted Normalizing Flow towards Arbitrary Stochastic Policy with Computational Efficiencycited by 1
  12. 2025Neural Cone Radiosity for Interactive Global Illumination with Glossy Materialscited by 1
7 more
  1. 2025DL101 Neural Network Outputs and Loss Functionscited by 1
  2. 2026Pseudo-Quantized Actor-Critic Algorithm for Robustness to Noisy Temporal Difference Errorcited by 1
  3. 2026sLTN: Structural Logic Tensor Networkscited by 1
  4. 2026Gradient Networks for Universal Magnetic Modeling of Synchronous Machinescited by 0
  5. 2026IGLU: The Integrated Gaussian Linear Unit Activation Functioncited by 0
  6. 2026Implementation of Hyperelastic Physics-Augmented Neural Networks in the Explicit Finite Element Codes Simcenter Radioss and OpenRadioss with Applications to Impact Eventscited by 0
  7. 2026The Spectral Neuroncited by 0