Task-Driven Kernel Flows: Label Rank Compression and Laplacian Spectral Filtering
Not on the map. Why?
Abstract (arXiv)
We present a theory of feature learning in wide L2-regularized networks showing that supervised learning is inherently compressive. We derive a kernel ODE that predicts a "water-filling" spectral evolution and prove that for any stable steady state, the kernel rank is bounded by the number of classes (C). We further demonstrate that SGD noise is similarly low-rank (O(C)), confining dynamics to the task-relevant subspace. This framework unifies the deterministic and stochastic views of alignment and contrasts the low-rank nature of supervised learning with the high-rank, expansive representations of self-supervision.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 9 of its 13 references that are in the map. First posted on arXiv 1 Jan 2026.
Builds on
9 of 13 references.
References · 9
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,597
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2018On the Spectral Bias of Neural Networkscited by 1,321
- 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
- 2018On Lazy Training in Differentiable Programmingcited by 671
- 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
- 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
- 2021Neural Collapse Under MSE Loss: Proximity to and Dynamics on the Central Pathcited by 156
Built on by
Citing papers per year: 2026: 1.
1 paper.