# A Compositional Kernel Model for Feature Learning

arXiv: https://arxiv.org/abs/2509.14158 · HTML: https://mlsanity.com/paper/2509.14158/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 36 references that are in the map.
- First posted on arXiv 17 Sep 2025.

## Builds on (most cited first, 9 of 9)

- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Kernel Feature Selection via Conditional Covariance Minimization](https://mlsanity.com/paper/1707.01164/) (2017)
- [xRFM: Accurate, scalable, and interpretable feature learning models for tabular data](https://mlsanity.com/paper/2508.10053/) (2025)
- [Position: A Theory of Deep Learning Must Include Compositional Sparsity](https://mlsanity.com/paper/2507.02550/) (2025)
- [Iteratively reweighted kernel machines efficiently learn sparse functions](https://mlsanity.com/paper/2505.08277/) (2025)
- [Enhanced Feature Learning via Regularisation: Integrating Neural Networks and Kernel Methods](https://mlsanity.com/paper/2407.17280/) (2024)
- [On the Self-Penalization Phenomenon in Feature Selection](https://mlsanity.com/paper/2110.05852/) (2021)
- [Taming Nonconvexity in Kernel Feature Selection -- Favorable Properties of the Laplace Kernel](https://mlsanity.com/paper/2106.09387/) (2021)
- [Gradient flow in the kernel learning problem](https://mlsanity.com/paper/2506.08550/) (2025)

## Built on by (most cited first, 1 of 1)

- [Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models](https://mlsanity.com/paper/2605.15082/) (2026)

## Abstract (arXiv)

We study a compositional variant of kernel ridge regression in which the predictor is applied to a coordinate-wise reweighting of the inputs. Formulated as a variational problem, this model provides a tractable setting for studying feature learning in compositional architectures. From the perspective of variable selection, we show how relevant variables are recovered while noise variables are eliminated. We prove that both global minimizers and stationary points discard noise coordinates when the noise variables are Gaussian distributed. A central finding is that ℓ1-type kernels, such as the Laplace kernel, succeed in recovering features contributing to nonlinear effects at stationary points, whereas Gaussian kernels recover only linear ones.