# Estimating the Spectral Density of Large Implicit Matrices

arXiv: https://arxiv.org/abs/1802.03451 · HTML: https://mlsanity.com/paper/1802.03451/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Feb 2018.

## Built on by (most cited first, 16 of 16)

- [Why Transformers Need Adam: A Hessian Perspective](https://mlsanity.com/paper/2402.16788/) (2024)
- [Hessian Eigenspectra of More Realistic Nonlinear Models](https://mlsanity.com/paper/2103.01519/) (2021)
- [Preconditioning for Scalable Gaussian Process Hyperparameter Optimization](https://mlsanity.com/paper/2107.00243/) (2021)
- [ViViT: Curvature access through the generalized Gauss-Newton's low-rank structure](https://mlsanity.com/paper/2106.02624/) (2021)
- [A Defense of the Quadratic Model](https://mlsanity.com/paper/2607.21716/) (2026)
- [Sparse Flows: Pruning Continuous-depth Models](https://mlsanity.com/paper/2106.12718/) (2021)
- [Flow Actor-Critic for Offline Reinforcement Learning](https://mlsanity.com/paper/2602.18015/) (2026)
- [HollowFlow: Efficient Sample Likelihood Evaluation using Hollow Message Passing](https://mlsanity.com/paper/2510.21542/) (2025)
- [Estimating Shape Distances on Neural Representations with Limited Samples](https://mlsanity.com/paper/2310.05742/) (2023)
- [Diffusion Density Estimators](https://mlsanity.com/paper/2410.06986/) (2024)
- [Spectral Estimation with Free Decompression](https://mlsanity.com/paper/2506.11994/) (2025)
- [Variance Reduction for Matrix Computations with Applications to Gaussian Processes](https://mlsanity.com/paper/2106.14565/) (2021)
- [Enhanced Distribution Modelling via Augmented Architectures For Neural ODE Flows](https://mlsanity.com/paper/2306.02731/) (2023)
- [Geometry-Aware Normalizing Wasserstein Flows for Optimal Causal Inference](https://mlsanity.com/paper/2311.18826/) (2023)
- [It's all In the (Exponential) Family: An Equivalence between Maximum Likelihood Estimation and Control Variates for Sketching Algorithms](https://mlsanity.com/paper/2601.22378/) (2026)
- [StAD: Stein Amortized Divergence for Fast Likelihoods with Diffusion and Flow](https://mlsanity.com/paper/2605.16486/) (2026)

## Abstract (arXiv)

Many important problems are characterized by the eigenvalues of a large matrix. For example, the difficulty of many optimization problems, such as those arising from the fitting of large models in statistics and machine learning, can be investigated via the spectrum of the Hessian of the empirical loss function. Network data can be understood via the eigenstructure of a graph Laplacian matrix using spectral graph theory. Quantum simulations and other many-body problems are often characterized via the eigenvalues of the solution space, as are various dynamic systems. However, naive eigenvalue estimation is computationally expensive even when the matrix can be represented; in many of these situations the matrix is so large as to only be available implicitly via products with vectors. Even worse, one may only have noisy estimates of such matrix vector products. In this work, we combine several different techniques for randomized estimation and show that it is possible to construct unbiased estimators to answer a broad class of questions about the spectra of such implicit matrices, even in the presence of noise. We validate these methods on large-scale problems in which graph theory and random matrix theory provide ground truth.