mlsanityData to 7 Oct 2026

Norm-preserving Orthogonal Permutation Linear Unit Activation Functions (OPLU)

arXiv 1604.02313 ↗First posted 8 Apr 2016cs.NEcited by 15

Not on the map. Why?

Abstract (arXiv)

We propose a novel activation function that implements piece-wise orthogonal non-linear mappings based on permutations. It is straightforward to implement, and very computationally efficient, also it has little memory requirements. We tested it on two toy problems for feedforward and recurrent networks, it shows similar performance to tanh and ReLU. OPLU activation function ensures norm preservance of the backpropagated gradients, therefore it is potentially good for the training of deep, extra deep, and recurrent neural networks.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). First posted on arXiv 8 Apr 2016.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 7; 2023: 3; 2024: 1; 2025: 2; 2026: 1.

15 papers.

Sort:

  1. 2022A Simple Approach to Improve Single-Model Deep Uncertainty via Distance-Awarenesscited by 49
  2. 2022Rethinking Lipschitz Neural Networks and Certified Robustness: A Boolean Function Perspectivecited by 48
  3. 2022Almost-Orthogonal Layers for Efficient General-Purpose Lipschitz Networkscited by 39
  4. 2022Kantorovich Strikes Back! Wasserstein GANs are not Optimal Transport?cited by 19
  5. 2022Approximation of Lipschitz Functions using Deep Spline Neural Networkscited by 15
  6. 2023Normalization-Equivariant Neural Networks with Application to Image Denoisingcited by 11
  7. 2025Approximation theory for 1-Lipschitz ResNetscited by 7
  8. 2023Convolutional Neural Networks as 2-D systemscited by 5
  9. 2025Enhancing Certified Robustness via Block Reflector Orthogonal Layers and Logit Annealing Losscited by 5
  10. 2022Robust-by-Design Classification via Unitary-Gradient Neural Networkscited by 4
  11. 2024A Wasserstein perspective of Vanilla GANscited by 3
  12. 2021Logical Activation Functions: Logit-space equivalents of Probabilistic Boolean Operatorscited by 2
3 more
  1. 2023Computing the Distance between unbalanced Distributions -- The flat Metriccited by 1
  2. 2026Preserving Plasticity in Continual Learning via Dynamical Isometrycited by 1
  3. 2022Adaptive n-ary Activation Functions for Probabilistic Boolean Logiccited by 0