# Local EGOP for Continuous Index Learning

arXiv: https://arxiv.org/abs/2601.07061 · HTML: https://mlsanity.com/paper/2601.07061/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 84 references that are in the map.
- First posted on arXiv 11 Jan 2026.

## Builds on (most cited first, 27 of 27)

- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Revisiting Deep Learning Models for Tabular Data](https://mlsanity.com/paper/2106.11959/) (2021)
- [To understand deep learning we need to understand kernel learning](https://mlsanity.com/paper/1802.01396/) (2018)
- [On Embeddings for Numerical Features in Tabular Deep Learning](https://mlsanity.com/paper/2203.05556/) (2022)
- [The merged-staircase property: a necessary and nearly sufficient condition for SGD learning of sparse functions on two-layer neural networks](https://mlsanity.com/paper/2202.08658/) (2022)
- [Learning Single-Index Models with Shallow Neural Networks](https://mlsanity.com/paper/2210.15651/) (2022)
- [Neural Networks Efficiently Learn Low-Dimensional Representations with SGD](https://mlsanity.com/paper/2209.14863/) (2022)
- [Smoothing the Landscape Boosts the Signal for SGD: Optimal Sample Complexity for Learning Single Index Models](https://mlsanity.com/paper/2305.10633/) (2023)
- [Transformers learn through gradual rank increase](https://mlsanity.com/paper/2306.07042/) (2023)
- [Neural network learns low-dimensional polynomials with SGD near the information-theoretic limit](https://mlsanity.com/paper/2406.01581/) (2024)
- [Repetita Iuvant: Data Repetition Allows SGD to Learn High-Dimensional Multi-Index Functions](https://mlsanity.com/paper/2405.15459/) (2024)
- [Besov Function Approximation and Binary Classification on Low-Dimensional Manifolds Using Convolutional Residual Networks](https://mlsanity.com/paper/2109.02832/) (2021)
- [Mechanism of feature learning in deep fully connected networks and kernel machines that recursively learn features](https://mlsanity.com/paper/2212.13881/) (2022)
- [A Geometric Perspective on Variational Autoencoders](https://mlsanity.com/paper/2209.07370/) (2022)
- [Hardness of Learning Neural Networks under the Manifold Hypothesis](https://mlsanity.com/paper/2406.01461/) (2024)
- [Mechanism of feature learning in convolutional neural networks](https://mlsanity.com/paper/2309.00570/) (2023)
- [Linear Recursive Feature Machines provably recover low-rank matrices](https://mlsanity.com/paper/2401.04553/) (2024)
- [xRFM: Accurate, scalable, and interpretable feature learning models for tabular data](https://mlsanity.com/paper/2508.10053/) (2025)
- [Iteratively reweighted kernel machines efficiently learn sparse functions](https://mlsanity.com/paper/2505.08277/) (2025)
- [Manifold Coordinates with Physical Meaning](https://mlsanity.com/paper/1811.11891/) (2018)
- [Robust Inference of Manifold Density and Geometry by Doubly Stochastic Scaling](https://mlsanity.com/paper/2209.08004/) (2022)
- [Efficient Estimation of the Central Mean Subspace via Smoothed Gradient Outer Products](https://mlsanity.com/paper/2312.15469/) (2023)
- [Coreset selection for the Sinkhorn divergence and generic smooth divergences](https://mlsanity.com/paper/2504.20194/) (2025)
- [Supervised Kernel Thinning](https://mlsanity.com/paper/2410.13749/) (2024)
- [LaSalle Invariance Principle for Discrete-time Dynamical Systems: A Concise and Self-contained Tutorial](https://mlsanity.com/paper/1710.03710/) (2017)
- [Conditional regression for the Nonlinear Single-Variable Model](https://mlsanity.com/paper/2411.09686/) (2024)
- [Surrogate to Poincar'e inequalities on manifolds for dimension reduction in nonlinear feature spaces](https://mlsanity.com/paper/2505.01807/) (2025)

## Built on by (most cited first, 1 of 1)

- [Average Gradient Outer Product in kernel regression provably recovers the central subspace for multi-index models](https://mlsanity.com/paper/2605.15082/) (2026)

## Abstract (arXiv)

We introduce the setting of continuous index learning, in which a function of many variables varies only along a small number of directions at each point. For efficient estimation, it is beneficial for a learning algorithm to adapt, near each point x, to the subspace that captures the local variability of the function f. We pose this task as kernel adaptation along a manifold with noise, and introduce Local EGOP learning, a recursive algorithm that utilizes the Expected Gradient Outer Product (EGOP) quadratic form as both a metric and inverse-covariance of our target distribution. We prove that Local EGOP learning adapts to the regularity of the function of interest, showing that under a supervised noisy manifold hypothesis, intrinsic dimensional learning rates are achieved for arbitrarily high-dimensional noise. Empirically, we compare our algorithm to the feature learning capabilities of deep learning. Additionally, we demonstrate improved regression quality compared to two-layer neural networks in the continuous single-index setting.