A Tunable Loss Function for Binary Classification

We present α-loss, α ∈ [1,∞], a tunable loss function for binary classification that bridges log-loss (α=1) and 0-1 loss (α = ∞). We prove that α-loss has an equivalent margin-based form and is classification-calibrated, two desirable properties for a good surrogate loss function for the ideal yet intractable 0-1 loss. For logistic regression-based classification, we provide an upper bound on the difference between the empirical and expected risk at the empirical risk minimizers for α-loss by exploiting its Lipschitzianity along with recent results on the landscape features of empirical risk functions. Finally, we show that α-loss with α = 2 performs better than log-loss on MNIST for logistic regression.