# A parametric activation function based on Wendland RBF

arXiv: https://arxiv.org/abs/2507.11493 · HTML: https://mlsanity.com/paper/2507.11493/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 28 references that are in the map.
- First posted on arXiv 28 Jun 2025.

## Builds on (most cited first, 15 of 15)

- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Empirical Evaluation of Rectified Activations in Convolutional Network](https://mlsanity.com/paper/1505.00853/) (2015)
- [Activation Functions in Deep Learning: A Comprehensive Survey and Benchmark](https://mlsanity.com/paper/2109.14545/) (2021)
- [Activation Functions: Comparison of trends in Practice and Research for Deep Learning](https://mlsanity.com/paper/1811.03378/) (2018)
- [On the Impact of the Activation Function on Deep Neural Networks Training](https://mlsanity.com/paper/1902.06853/) (2019)
- [A survey on modern trainable activation functions](https://mlsanity.com/paper/2005.00817/) (2020)
- [Pad'e Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks](https://mlsanity.com/paper/1907.06732/) (2019)
- [Noisy Activation Functions](https://mlsanity.com/paper/1603.00391/) (2016)
- [Learning Activation Functions: A new paradigm for understanding Neural Networks](https://mlsanity.com/paper/1906.09529/) (2019)
- [Revise Saturated Activation Functions](https://mlsanity.com/paper/1602.05980/) (2016)
- [A Method on Searching Better Activation Functions](https://mlsanity.com/paper/2405.12954/) (2024)
- [Saturated Non-Monotonic Activation Functions](https://mlsanity.com/paper/2305.07537/) (2023)
- [Efficient Search for Customized Activation Functions with Gradient Descent](https://mlsanity.com/paper/2408.06820/) (2024)

## Abstract (arXiv)

This paper introduces a novel parametric activation function based on Wendland radial basis functions (RBFs) for deep neural networks. Wendland RBFs, known for their compact support, smoothness, and positive definiteness in approximation theory, are adapted to address limitations of traditional activation functions like ReLU, sigmoid, and tanh. The proposed enhanced Wendland activation combines a standard Wendland component with linear and exponential terms, offering tunable locality, improved gradient propagation, and enhanced stability during training. Theoretical analysis highlights its mathematical properties, including smoothness and adaptability, while empirical experiments on synthetic tasks (e.g., sine wave approximation) and benchmark datasets (MNIST, Fashion-MNIST) demonstrate competitive performance. Results show that the Wendland-based activation achieves superior accuracy in certain scenarios, particularly in regression tasks, while maintaining computational efficiency. The study bridges classical RBF theory with modern deep learning, suggesting that Wendland activations can mitigate overfitting and improve generalization through localized, smooth transformations. Future directions include hybrid architectures and domain-specific adaptations.