# On Data-Dependent Random Features for Improved Generalization in Supervised Learning

arXiv: https://arxiv.org/abs/1712.07102 · HTML: https://mlsanity.com/paper/1712.07102/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Dec 2017.

## Built on by (most cited first, 4 of 4)

- [Optimal Nonlinearities Improve Generalization Performance of Random Features](https://mlsanity.com/paper/2309.16846/) (2023)
- [On The Relative Error of Random Fourier Features for Preserving Kernel Distance](https://mlsanity.com/paper/2210.00244/) (2022)
- [All Random Features Representations are Equivalent](https://mlsanity.com/paper/2406.18802/) (2024)
- [Decentralized Kernel Ridge Regression Based on Data-Dependent Random Feature](https://mlsanity.com/paper/2405.07791/) (2024)

## Abstract (arXiv)

The randomized-feature approach has been successfully employed in large-scale kernel approximation and supervised learning. The distribution from which the random features are drawn impacts the number of features required to efficiently perform a learning task. Recently, it has been shown that employing data-dependent randomization improves the performance in terms of the required number of random features. In this paper, we are concerned with the randomized-feature approach in supervised learning for good generalizability. We propose the Energy-based Exploration of Random Features (EERF) algorithm based on a data-dependent score function that explores the set of possible features and exploits the promising regions. We prove that the proposed score function with high probability recovers the spectrum of the best fit within the model class. Our empirical results on several benchmark datasets further verify that our method requires smaller number of random features to achieve a certain generalization error compared to the state-of-the-art while introducing negligible pre-processing overhead. EERF can be implemented in a few lines of code and requires no additional tuning parameters.