# SpecNet2: Orthogonalization-free spectral embedding by neural networks

arXiv: https://arxiv.org/abs/2206.06644 · HTML: https://mlsanity.com/paper/2206.06644/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 14 Jun 2022.

## Builds on (most cited first, 8 of 8)

- [SpectralNet: Spectral Clustering using Deep Neural Networks](https://mlsanity.com/paper/1801.01587/) (2018)
- [Error estimates for spectral convergence of the graph Laplacian on random geometric graphs towards the Laplace--Beltrami operator](https://mlsanity.com/paper/1801.10108/) (2018)
- [Lipschitz regularity of graph Laplacians on random data clouds](https://mlsanity.com/paper/2007.06679/) (2020)
- [Eigen-convergence of Gaussian kernelized graph Laplacian by manifold heat interpolation](https://mlsanity.com/paper/2101.09875/) (2021)
- [Spectral Convergence of the connection Laplacian from random samples](https://mlsanity.com/paper/1306.1587/) (2013)
- [Diffusion Nets](https://mlsanity.com/paper/1506.07840/) (2015)
- [Improved spectral convergence rates for graph Laplacians on epsilon-graphs and k-NN graphs](https://mlsanity.com/paper/1910.13476/) (2019)
- [Convergence of Graph Laplacian with kNN Self-tuned Kernels](https://mlsanity.com/paper/2011.01479/) (2020)

## Built on by (most cited first, 5 of 5)

- [Graph Laplacian for Semi-Supervised Learning](https://mlsanity.com/paper/2301.04956/) (2023)
- [BASiS: Batch Aligned Spectral Embedding Space](https://mlsanity.com/paper/2211.16960/) (2022)
- [Neural Normalized Cut: A Differential and Generalizable Approach for Spectral Clustering](https://mlsanity.com/paper/2503.09260/) (2025)
- [Generalizable Spectral Embedding with an Application to UMAP](https://mlsanity.com/paper/2501.11305/) (2025)
- [A Graph Laplacian Eigenvector-based Pre-training Method for Graph Neural Networks](https://mlsanity.com/paper/2509.02803/) (2025)

## Abstract (arXiv)

Spectral methods which represent data points by eigenvectors of kernel matrices or graph Laplacian matrices have been a primary tool in unsupervised data analysis. In many application scenarios, parametrizing the spectral embedding by a neural network that can be trained over batches of data samples gives a promising way to achieve automatic out-of-sample extension as well as computational scalability. Such an approach was taken in the original paper of SpectralNet (Shaham et al. 2018), which we call SpecNet1. The current paper introduces a new neural network approach, named SpecNet2, to compute spectral embedding which optimizes an equivalent objective of the eigen-problem and removes the orthogonalization layer in SpecNet1. SpecNet2 also allows separating the sampling of rows and columns of the graph affinity matrix by tracking the neighbors of each data point through the gradient formula. Theoretically, we show that any local minimizer of the new orthogonalization-free objective reveals the leading eigenvectors. Furthermore, global convergence for this new orthogonalization-free objective using a batch-based gradient descent method is proved. Numerical experiments demonstrate the improved performance and computational efficiency of SpecNet2 on simulated data and image datasets.