mlsanityData to 7 Oct 2026

Nish: A Novel Negative Stimulated Hybrid Activation Function

arXiv 2210.09083 ↗First posted 17 Oct 2022cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

An activation function has a significant impact on the efficiency and robustness of the neural networks. As an alternative, we evolved a cutting-edge non-monotonic activation function, Negative Stimulated Hybrid Activation Function (Nish). It acts as a Rectified Linear Unit (ReLU) function for the positive region and a sinus-sigmoidal function for the negative region. In other words, it incorporates a sigmoid and a sine function and gaining new dynamics over classical ReLU. We analyzed the consistency of the Nish for different combinations of essential networks and most common activation functions using on several most popular benchmarks. From the experimental results, we reported that the accuracy rates achieved by the Nish is slightly better than compared to the Mish in classification.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 21 of its 36 references that are in the map. First posted on arXiv 17 Oct 2022.

Builds on

21 of 36 references.

References · 21

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  5. 2017Squeeze-and-Excitation Networkscited by 4,763
  6. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2016Wide Residual Networkscited by 3,057
  9. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  10. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  11. 2015Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)cited by 1,061
  12. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
9 more
  1. 2014Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognitioncited by 719
  2. 2017Self-Normalizing Neural Networkscited by 591
  3. 2015Empirical Evaluation of Rectified Activations in Convolutional Networkcited by 515
  4. 2013OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networkscited by 224
  5. 2018Activation Functions: Comparison of trends in Practice and Research for Deep Learningcited by 130
  6. 2020Evolving Normalization-Activation Layerscited by 45
  7. 2018Collapse of Deep and Narrow Neural Netscited by 37
  8. 2020Evolutionary Optimization of Deep Learning Activation Functionscited by 18
  9. 2018The Quest for the Golden Activation Functioncited by 14

Built on by

Not cited yet.