# The Galerkin method beats Graph-Based Approaches for Spectral Algorithms

arXiv: https://arxiv.org/abs/2306.00742 · HTML: https://mlsanity.com/paper/2306.00742/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 68 references that are in the map.
- First posted on arXiv 1 Jun 2023.

## Builds on (most cited first, 25 of 25)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learning](https://mlsanity.com/paper/2105.04906/) (2021)
- [Convex Optimization: Algorithms and Complexity](https://mlsanity.com/paper/1405.4980/) (2014)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [An Introduction to Matrix Concentration Inequalities](https://mlsanity.com/paper/1501.01571/) (2015)
- [Provable Guarantees for Self-Supervised Deep Learning with Spectral Contrastive Loss](https://mlsanity.com/paper/2106.04156/) (2021)
- [Whitening for Self-Supervised Representation Learning](https://mlsanity.com/paper/2007.06346/) (2020)
- [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021)
- [Generalization Properties of Learning with Random Features](https://mlsanity.com/paper/1602.04474/) (2016)
- [Predicting What You Already Know Helps: Provable Self-Supervised Learning](https://mlsanity.com/paper/2008.01064/) (2020)
- [Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods](https://mlsanity.com/paper/2205.11508/) (2022)
- [Less is More: Nystr"om Computational Regularization](https://mlsanity.com/paper/1507.04717/) (2015)
- [Kernel methods through the roof: handling billions of points efficiently](https://mlsanity.com/paper/2006.10350/) (2020)
- [The SSL Interplay: Augmentations, Inductive Bias, and Generalization](https://mlsanity.com/paper/2302.02774/) (2023)
- [Machine learning in acoustics: theory and applications](https://mlsanity.com/paper/1905.04418/) (2019)
- [Solving high-dimensional eigenvalue problems using deep neural networks: A diffusion Monte Carlo like approach](https://mlsanity.com/paper/2002.02600/) (2020)
- [Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functions](https://mlsanity.com/paper/2210.01883/) (2022)
- [Contrastive Laplacian Eigenmaps](https://mlsanity.com/paper/2201.05493/) (2022)
- [Neural Eigenfunctions Are Structured Representation Learners](https://mlsanity.com/paper/2210.12637/) (2022)
- [Overcoming the curse of dimensionality with Laplacian regularization in semi-supervised learning](https://mlsanity.com/paper/2009.04324/) (2020)
- [Solving eigenvalue PDEs of metastable diffusion processes using artificial neural networks](https://mlsanity.com/paper/2110.14523/) (2021)
- [On minimal variations for unsupervised representation learning](https://mlsanity.com/paper/2211.03782/) (2022)
- [Kernelized Diffusion maps](https://mlsanity.com/paper/2302.06757/) (2023)
- [How many samples are needed to leverage smoothness?](https://mlsanity.com/paper/2305.16014/) (2023)

## Built on by (most cited first, 6 of 6)

- [Learning the Infinitesimal Generator of Stochastic Diffusion Processes](https://mlsanity.com/paper/2405.12940/) (2024)
- [Laplace Transform Based Low-Complexity Learning of Continuous Markov Semigroups](https://mlsanity.com/paper/2410.14477/) (2024)
- [From Biased to Unbiased Dynamics: An Infinitesimal Generator Approach](https://mlsanity.com/paper/2406.09028/) (2024)
- [A Schr"odinger Eigenfunction Method for Long-Horizon Stochastic Optimal Control](https://mlsanity.com/paper/2603.23173/) (2026)
- [Adaptive Kernel Selection for Kernelized Diffusion Maps](https://mlsanity.com/paper/2604.18402/) (2026)
- [Langevin-Informed Transfer Learning: Replacing Target Samples by Black-Box Feedback](https://mlsanity.com/paper/2610.01522/) (2026)

## Abstract (arXiv)

Historically, the machine learning community has derived spectral decompositions from graph-based approaches. We break with this approach and prove the statistical and computational superiority of the Galerkin method, which consists in restricting the study to a small set of test functions. In particular, we introduce implementation tricks to deal with differential operators in large dimensions with structured kernels. Finally, we extend on the core principles beyond our approach to apply them to non-linear spaces of functions, such as the ones parameterized by deep neural networks, through loss-based optimization procedures.