# Rank Is Not Capacity: Spectral Occupancy for Latent Graph Models

arXiv: https://arxiv.org/abs/2605.11142 · HTML: https://mlsanity.com/paper/2605.11142/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 66 references that are in the map.
- First posted on arXiv 11 May 2026.

## Builds on (most cited first, 29 of 29)

- [node2vec: Scalable Feature Learning for Networks](https://mlsanity.com/paper/1607.00653/) (2016)
- [DeepWalk: Online Learning of Social Representations](https://mlsanity.com/paper/1403.6652/) (2014)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Benign Overfitting in Linear Regression](https://mlsanity.com/paper/1906.11300/) (2019)
- [Implicit Regularization in Deep Matrix Factorization](https://mlsanity.com/paper/1905.13655/) (2019)
- [Measuring the Intrinsic Dimension of Objective Landscapes](https://mlsanity.com/paper/1804.08838/) (2018)
- [Implicit Regularization in Matrix Factorization](https://mlsanity.com/paper/1705.09280/) (2017)
- [The Vendi Score: A Diversity Evaluation Metric for Machine Learning](https://mlsanity.com/paper/2210.02410/) (2022)
- [Network Embedding as Matrix Factorization: Unifying DeepWalk, LINE, PTE, and node2vec](https://mlsanity.com/paper/1710.02971/) (2017)
- [Statistical inference on random dot product graphs: a survey](https://mlsanity.com/paper/1709.05454/) (2017)
- [A statistical interpretation of spectral embedding: the generalised random dot product graph](https://mlsanity.com/paper/1709.05506/) (2017)
- [Stable Rank Normalization for Improved Generalization in Neural Networks and GANs](https://mlsanity.com/paper/1906.04659/) (2019)
- [Diff-eRank: A Novel Rank-Based Metric for Evaluating Large Language Models](https://mlsanity.com/paper/2401.17139/) (2024)
- [Network cross-validation by edge sampling](https://mlsanity.com/paper/1612.04717/) (2016)
- [Network Cross-Validation for Determining the Number of Communities in Network Data](https://mlsanity.com/paper/1411.1715/) (2014)
- [Principled approach to the selection of the embedding dimension of networks](https://mlsanity.com/paper/2004.09928/) (2020)
- [Learning Role-based Graph Embeddings](https://mlsanity.com/paper/1802.02896/) (2018)
- [Graph Entropy Guided Node Embedding Dimension Selection for Graph Neural Networks](https://mlsanity.com/paper/2105.03178/) (2021)
- [A Hierarchical Block Distance Model for Ultra Low-Dimensional Graph Representations](https://mlsanity.com/paper/2204.05885/) (2022)
- [Characterizing Polarization in Social Networks using the Signed Relational Latent Distance Model](https://mlsanity.com/paper/2301.09507/) (2023)
- [Spectral Scaling Laws in Language Models: How Effectively Do Feed-Forward Networks Use Their Latent Space?](https://mlsanity.com/paper/2510.00537/) (2025)
- [HM-LDM: A Hybrid-Membership Latent Distance Model](https://mlsanity.com/paper/2206.03463/) (2022)
- [Bayesian estimation of the latent dimension and communities in stochastic blockmodels](https://mlsanity.com/paper/1904.05333/) (2019)
- [What Do GNNs Actually Learn? Towards Understanding their Representations](https://mlsanity.com/paper/2304.10851/) (2023)
- [Gradient-Based Spectral Embeddings of Random Dot Product Graphs](https://mlsanity.com/paper/2307.13818/) (2023)
- [How Low Can You Go? Searching for the Intrinsic Dimensionality of Complex Networks using Metric Node Embeddings](https://mlsanity.com/paper/2503.01723/) (2025)
- [Low-Rank Matrix Approximation for Neural Network Compression](https://mlsanity.com/paper/2504.20078/) (2025)
- [Aitchison Embeddings for Learning Compositional Graph Representations](https://mlsanity.com/paper/2605.00716/) (2026)
- [Network Signflip Parallel Analysis for Selecting the Embedding Dimension](https://mlsanity.com/paper/2509.05722/) (2025)

## Built on by (most cited first, 1 of 1)

- [Same Architecture, Different Capacity: Optimizer-Induced Spectral Scaling Laws](https://mlsanity.com/paper/2605.21803/) (2026)

## Abstract (arXiv)

Graph representation learning has become a standard approach for analyzing networked data, with latent embeddings widely used for link prediction, community detection, and related tasks. Yet a basic design choice, the latent dimension, is still treated as a brittle hyperparameter, fixed before training and tuned by held-out performance. Learned factors are also identifiable only up to rotation and rescaling, so the nominal rank rarely coincides with the quantity that governs model behavior. We propose Spectral Prefix Extraction and Capacity-Targeted Representation Analysis (Spectra), which replaces rank as the unit of analysis with the spectrum of a learned positive semidefinite kernel, trace-normalized so that spectra are comparable across fits. The normalized eigenvalues form a distribution on the simplex, and their Shannon effective rank acts both as a summary of learned capacity and as a controllable training-time coordinate: a single scalar shapes this realized dimension during training, and bisection targets any desired value within the rank cap. To theoretically support that, we show local regularity and monotonicity of the realized-dimension profile. Across collaboration, social, biological, and infrastructure networks, Spectra traces performance--capacity frontiers that make the trade-off between predictive accuracy and realized dimension visible. It performs competitively with strong link-prediction baselines, yields aligned lower-capacity views of the same fitted model through spectral prefixes, and provides a principled handle on capacity in the overparameterized regime. Capacity thus becomes a property of the fitted model rather than a hyperparameter of the training.