# Binary Kernel Logistic Regression: a sparsity-inducing formulation and a convergent decomposition training algorithm

arXiv: https://arxiv.org/abs/2512.19440 · HTML: https://mlsanity.com/paper/2512.19440/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 6 of its 60 references that are in the map.
- First posted on arXiv 22 Dec 2025.

## Builds on (most cited first, 6 of 6)

- [Parsimonious Online Learning with Kernels via Sparse Projections in Function Space](https://mlsanity.com/paper/1612.04111/) (2016)
- [Scalable Kernel Logistic Regression with Nystr"om Approximation: Theoretical Analysis and Application to Discrete Choice Modelling](https://mlsanity.com/paper/2402.06763/) (2024)
- [Incremental Import Vector Machines for Classifying Hyperspectral Data](https://mlsanity.com/paper/1708.05966/) (2017)
- [Cost-sensitive probabilistic predictions for support vector machines](https://mlsanity.com/paper/2310.05997/) (2023)
- [Sparse Learning and Class Probability Estimation with Weighted Support Vector Machines](https://mlsanity.com/paper/2312.10618/) (2023)
- [Soft regression trees: a model variant and a decomposition training algorithm](https://mlsanity.com/paper/2501.05942/) (2025)

## Abstract (arXiv)

Kernel logistic regression (KLR) is a widely used supervised learning method for binary and multi-class classification, which provides estimates of the conditional probabilities of class membership for the data points. Unlike other kernel methods such as Support Vector Machines (SVMs), KLRs are generally not sparse. Previous attempts to deal with sparsity in KLR include a heuristic method referred to as the Import Vector Machine (IVM) and ad hoc regularizations such as the ℓ1/2-based one. Achieving a good trade-off between prediction accuracy and sparsity is still a challenging issue with a potential significant impact from the application point of view. In this work, we revisit binary KLR and propose an extension of the training formulation proposed by Keerthi et al., which is able to induce sparsity in the trained model, while maintaining good testing accuracy. To efficiently solve the dual of this formulation, we devise a decomposition algorithm of Sequential Minimal Optimization type which exploits second-order information, and for which we establish global convergence. Numerical experiments conducted on 12 datasets from the literature show that the proposed binary KLR approach achieves a competitive trade-off between accuracy and sparsity with respect to IVM, ℓ1/2-based regularization for KLR, and SVM while retaining the advantages of providing informative estimates of the class membership probabilities.