# Neural Feature Learning in Function Space

arXiv: https://arxiv.org/abs/2309.10140 · HTML: https://mlsanity.com/paper/2309.10140/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Sep 2023.

## Built on by (most cited first, 12 of 12)

- [From Lazy to Rich: Exact Learning Dynamics in Deep Linear Networks](https://mlsanity.com/paper/2409.14623/) (2024)
- [Operator SVD with Neural Networks via Nested Low-Rank Approximation](https://mlsanity.com/paper/2402.03655/) (2024)
- [Efficient Parametric SVD of Koopman Operator for Stochastic Dynamical Systems](https://mlsanity.com/paper/2507.07222/) (2025)
- [Information-Theoretic Measures on Lattices for High-Order Interactions](https://mlsanity.com/paper/2408.07533/) (2024)
- [Dependence Induced Representations](https://mlsanity.com/paper/2411.15328/) (2024)
- [Adversarial Dependence Minimization](https://mlsanity.com/paper/2502.03227/) (2025)
- [Separable Computation of Information Measures](https://mlsanity.com/paper/2501.15301/) (2025)
- [Revisiting Orbital Minimization Method for Neural Operator Decomposition](https://mlsanity.com/paper/2510.21952/) (2025)
- [Eigenfunction Extraction for Ordered Representation Learning](https://mlsanity.com/paper/2510.24672/) (2025)
- [A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis](https://mlsanity.com/paper/2603.11428/) (2026)
- [Fourier Preconditioning for Neural Feature Learning](https://mlsanity.com/paper/2607.02199/) (2026)
- [Universal Distributed Feature Extraction Complementary to Side Information via H-Score](https://mlsanity.com/paper/2610.16707/) (2026)

## Abstract (arXiv)

We present a novel framework for learning system design with neural feature extractors. First, we introduce the feature geometry, which unifies statistical dependence and feature representations in a function space equipped with inner products. This connection defines function-space concepts on statistical dependence, such as norms, orthogonal projection, and spectral decomposition, exhibiting clear operational meanings. In particular, we associate each learning setting with a dependence component and formulate learning tasks as finding corresponding feature approximations. We propose a nesting technique, which provides systematic algorithm designs for learning the optimal features from data samples with off-the-shelf network architectures and optimizers. We further demonstrate multivariate learning applications, including conditional inference and multimodal learning, where we present the optimal features and reveal their connections to classical approaches.