ULU: A Unified Activation Function

We propose ULU, a novel non-monotonic, piecewise activation function defined as f(x;α1),x<0; f(x;α2),x>=0 , where f(x;α)=0.5x(tanh(α x)+1),α >0. ULU treats positive and negative inputs differently. Extensive experiments demonstrate ULU significantly outperforms ReLU and Mish across image classification and object detection tasks. Its variant Adaptive ULU (AULU) is expressed as f(x;β12),x<0; f(x;β22),x>=0 , where β1 and β2 are learnable parameters, enabling it to adapt its response separately for positive and negative inputs. Additionally, we introduce the LIB (Like Inductive Bias) metric from AULU to quantitatively measure the inductive bias of the model.