# Spectral Inference Networks: Unifying Deep and Spectral Learning

arXiv: https://arxiv.org/abs/1806.02215 · HTML: https://mlsanity.com/paper/1806.02215/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2018.

## Built on by (most cited first, 30 of 35)

- [Flow Matching on General Geometries](https://mlsanity.com/paper/2302.03660/) (2023)
- [Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond](https://mlsanity.com/paper/2101.11517/) (2021)
- [Contrastive and Non-Contrastive Self-Supervised Learning Recover Global and Local Spectral Embedding Methods](https://mlsanity.com/paper/2205.11508/) (2022)
- [Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks](https://mlsanity.com/paper/2304.12567/) (2023)
- [Contrastive Learning Can Find An Optimal Basis For Approximately View-Invariant Functions](https://mlsanity.com/paper/2210.01883/) (2022)
- [Deep Laplacian-based Options for Temporally-Extended Exploration](https://mlsanity.com/paper/2301.11181/) (2023)
- [NeuralEF: Deconstructing Kernels by Deep Neural Networks](https://mlsanity.com/paper/2205.00165/) (2022)
- [Towards Better Laplacian Representation in Reinforcement Learning with Generalized Graph Drawing](https://mlsanity.com/paper/2107.05545/) (2021)
- [Neural Eigenfunctions Are Structured Representation Learners](https://mlsanity.com/paper/2210.12637/) (2022)
- [The Edge of Orthogonality: A Simple View of What Makes BYOL Tick](https://mlsanity.com/paper/2302.04817/) (2023)
- [Operator SVD with Neural Networks via Nested Low-Rank Approximation](https://mlsanity.com/paper/2402.03655/) (2024)
- [Spectral Representation Learning for Conditional Moment Models](https://mlsanity.com/paper/2210.16525/) (2022)
- [Proper Laplacian Representation Learning](https://mlsanity.com/paper/2310.10833/) (2023)
- [Occam's Razor for Self Supervised Learning: What is Sufficient to Learn Good Representations?](https://mlsanity.com/paper/2406.10743/) (2024)
- [A Novel Stochastic Gradient Descent Algorithm for Learning Principal Subspaces](https://mlsanity.com/paper/2212.04025/) (2022)
- [Harnessing small projectors and multiple views for efficient vision pretraining](https://mlsanity.com/paper/2312.10725/) (2023)
- [Unsupervised Learning on a DIET: Datum IndEx as Target Free of Self-Supervision, Reconstruction, Projector Head](https://mlsanity.com/paper/2302.10260/) (2023)
- [Learning Neural Eigenfunctions for Unsupervised Semantic Segmentation](https://mlsanity.com/paper/2304.02841/) (2023)
- [Network Representation Learning: From Traditional Feature Learning to Deep Learning](https://mlsanity.com/paper/2103.04339/) (2021)
- [STNet: Spectral Transformation Network for Solving Operator Eigenvalue Problem](https://mlsanity.com/paper/2510.23986/) (2025)
- [Spectal Harmonics: Bridging Spectral Embedding and Matrix Completion in Self-Supervised Learning](https://mlsanity.com/paper/2305.19818/) (2023)
- [Generalizable Spectral Embedding with an Application to UMAP](https://mlsanity.com/paper/2501.11305/) (2025)
- [Deep Clustering via Probabilistic Ratio-Cut Optimization](https://mlsanity.com/paper/2502.03405/) (2025)
- [Multilayer Graph Clustering with Optimized Node Embedding](https://mlsanity.com/paper/2103.16534/) (2021)
- [The Symmetric Generalized Eigenvalue Problem as a Nash Equilibrium](https://mlsanity.com/paper/2206.04993/) (2022)
- [Nash Equilibria via Stochastic Eigendecomposition](https://mlsanity.com/paper/2411.02308/) (2024)
- [Revisiting Orbital Minimization Method for Neural Operator Decomposition](https://mlsanity.com/paper/2510.21952/) (2025)
- [Eigenfunction Extraction for Ordered Representation Learning](https://mlsanity.com/paper/2510.24672/) (2025)
- [Learning-Guided Integration Contours Construction for Fast Large-Scale Generalized Eigensolvers](https://mlsanity.com/paper/2511.01927/) (2025)
- [NeuraLSP: A Neural Spectral Preconditioner for Accelerating PDE Solvers](https://mlsanity.com/paper/2601.20174/) (2026)

## Abstract (arXiv)

We present Spectral Inference Networks, a framework for learning eigenfunctions of linear operators by stochastic optimization. Spectral Inference Networks generalize Slow Feature Analysis to generic symmetric operators, and are closely related to Variational Monte Carlo methods from computational physics. As such, they can be a powerful tool for unsupervised representation learning from video or graph-structured data. We cast training Spectral Inference Networks as a bilevel optimization problem, which allows for online learning of multiple eigenfunctions. We show results of training Spectral Inference Networks on problems in quantum mechanics and feature learning for videos on synthetic datasets. Our results demonstrate that Spectral Inference Networks accurately recover eigenfunctions of linear operators and can discover interpretable representations from video in a fully unsupervised manner.