# Double Sparsity Kernel Learning with Automatic Variable Selection and Data Extraction

arXiv: https://arxiv.org/abs/1706.01426 · HTML: https://mlsanity.com/paper/1706.01426/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2017.

## Built on by (most cited first, 3 of 3)

- [Learning Acceptance Regions for Many Classes with Anomaly Detection](https://mlsanity.com/paper/2209.09963/) (2022)
- [Sparse deep neural networks for nonparametric estimation in high-dimensional sparse regression](https://mlsanity.com/paper/2406.18137/) (2024)
- [Nonparametric Envelopes for Flexible Response Reduction](https://mlsanity.com/paper/2509.07248/) (2025)

## Abstract (arXiv)

Learning with Reproducing Kernel Hilbert Spaces (RKHS) has been widely used in many scientific disciplines. Because a RKHS can be very flexible, it is common to impose a regularization term in the optimization to prevent overfitting. Standard RKHS learning employs the squared norm penalty of the learning function. Despite its success, many challenges remain. In particular, one cannot directly use the squared norm penalty for variable selection or data extraction. Therefore, when there exists noise predictors, or the underlying function has a sparse representation in the dual space, the performance of standard RKHS learning can be suboptimal. In the literature,work has been proposed on how to perform variable selection in RKHS learning, and a data sparsity constraint was considered for data extraction. However, how to learn in a RKHS with both variable selection and data extraction simultaneously remains unclear. In this paper, we propose a unified RKHS learning method, namely, DOuble Sparsity Kernel (DOSK) learning, to overcome this challenge. An efficient algorithm is provided to solve the corresponding optimization problem. We prove that under certain conditions, our new method can asymptotically achieve variable selection consistency. Simulated and real data results demonstrate that DOSK is highly competitive among existing approaches for RKHS learning.