# Enhanced Feature Learning via Regularisation: Integrating Neural Networks and Kernel Methods

arXiv: https://arxiv.org/abs/2407.17280 · HTML: https://mlsanity.com/paper/2407.17280/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jul 2024.

## Built on by (most cited first, 5 of 5)

- [Learning Multi-Index Models with Hyper-Kernel Ridge Regression](https://mlsanity.com/paper/2510.02532/) (2025)
- [A Compositional Kernel Model for Feature Learning](https://mlsanity.com/paper/2509.14158/) (2025)
- [On the Koopman-Based Generalization Bounds for Multi-Task Deep Learning](https://mlsanity.com/paper/2512.19199/) (2025)
- [How does feature learning reshape the function space?](https://mlsanity.com/paper/2605.17718/) (2026)
- [Featured Reproducing Kernel Banach Spaces for Learning and Neural Networks](https://mlsanity.com/paper/2602.07141/) (2026)

## Abstract (arXiv)

We propose a new method for feature learning and function estimation in supervised learning via regularised empirical risk minimisation. Our approach considers functions as expectations of Sobolev functions over all possible one-dimensional projections of the data. This framework is similar to kernel ridge regression, where the kernel is mathbbEw ( k(B)(w^top x,w^top x^prime)), with k(B)(a,b) := min(|a|, |b|)mathds1ab>0 the Brownian kernel, and the distribution of the projections w is learnt. This can also be viewed as an infinite-width one-hidden layer neural network, optimising the first layer's weights through gradient descent and explicitly adjusting the non-linearity and weights of the second layer. We introduce a gradient-based computational method for the estimator, called Brownian Kernel Neural Network (BKerNN), using particles to approximate the expectation, where the positive homogeneity of the Brownian kernel redleads to improved robustness to local minima. Using Rademacher complexity, we show that BKerNN's expected risk converges to the minimal risk with explicit high-probability rates of O( min((d/n)1/2, n-1/6)) (up to logarithmic factors). Numerical experiments confirm our optimisation intuitions, and BKerNN outperforms kernel ridge regression, and favourably compares to a one-hidden layer neural network with ReLU activations in various settings and real data sets.