# Learning with Covariance Matrices: Principal Component Analysis Meets Learning with Graphs

arXiv: https://arxiv.org/abs/2609.10490 · HTML: https://mlsanity.com/paper/2609.10490/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 59 references that are in the map.
- First posted on arXiv 9 Sep 2026.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Tangent Kernel: Convergence and Generalization in Neural Networks](https://mlsanity.com/paper/1806.07572/) (2018)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)
- [Stability Properties of Graph Neural Networks](https://mlsanity.com/paper/1905.04497/) (2019)
- [Connecting the Dots: Identifying Network Structure via Graph Signal Processing](https://mlsanity.com/paper/1810.13066/) (2018)
- [Graph Neural Networks: Architectures, Stability and Transferability](https://mlsanity.com/paper/2008.01767/) (2020)
- [Graphs, Convolutions, and Neural Networks: From Graph Filters to Graph Neural Networks](https://mlsanity.com/paper/2003.03777/) (2020)
- [Graph Filters for Signal Processing and Machine Learning on Graphs](https://mlsanity.com/paper/2211.08854/) (2022)
- [Graph Signal Processing: History, Development, Impact, and Outlook](https://mlsanity.com/paper/2303.12211/) (2023)
- [Graphon Signal Processing](https://mlsanity.com/paper/2003.05030/) (2020)
- [Transformers are Graph Neural Networks](https://mlsanity.com/paper/2506.22084/) (2025)
- [Sparse PCA via Covariance Thresholding](https://mlsanity.com/paper/1311.5179/) (2013)
- [coVariance Neural Networks](https://mlsanity.com/paper/2205.15856/) (2022)
- [Explainable Brain Age Prediction using coVariance Neural Networks](https://mlsanity.com/paper/2305.18370/) (2023)
- [Spatiotemporal Covariance Neural Networks](https://mlsanity.com/paper/2409.10068/) (2024)
- [Sparse Covariance Neural Networks](https://mlsanity.com/paper/2410.01669/) (2024)
- [Fair CoVariance Neural Networks](https://mlsanity.com/paper/2409.08558/) (2024)
- [Sparse PCA on fixed-rank matrices](https://mlsanity.com/paper/2201.02487/) (2022)
- [The curse of isotropy: from principal components to principal subspaces](https://mlsanity.com/paper/2307.15348/) (2023)
- [Disentangling Neurodegeneration with Brain Age Gap Prediction Models: A Graph Signal Processing Perspective](https://mlsanity.com/paper/2510.12763/) (2025)
- [Diagonally-Dominant Principal Component Analysis](https://mlsanity.com/paper/1906.00051/) (2019)

## Built on by (most cited first, 2 of 2)

- [Spatiotemporal Kronecker Covariance Neural Networks](https://mlsanity.com/paper/2609.25326/) (2026)
- [Joint Precision Neural Networks: Task-Aware Dependency and Predictive Learning](https://mlsanity.com/paper/2610.06023/) (2026)

## Abstract (arXiv)

This feature article provides an overview of the theoretical foundations for coVariance neural networks (VNNs), i.e., graph neural networks (GNNs) operating on covariance matrices as graphs. Covariance matrices are ubiquitous across domains, and hence, the deployment of GNNs often leverages graphs of pairwise statistical dependencies. Existing theoretical contributions on GNNs consider abstract graph representations and cannot accommodate the data-driven nuances associated with covariance matrices. This tutorial brings into focus various novel theoretical insights via mathematical analyses of VNNs that have broad signal processing implications, including: (i) a conceptual equivalence between VNNs and principal component analysis (PCA)-based information processing; (ii) refined stability bounds on predictive outcomes in the presence of finite sample-induced covariance matrix perturbations; and (iii) refined characterization of transferability of VNNs across multiscale datasets. The theoretical insights discussed herein provide the underlying principles and justification towards adopting VNNs over workhorse PCA-based learning pipelines, in applications where covariance matrices are useful descriptors of data structure. We also convey how impact of these foundational advances permeates to textitprincipled designs and applications of learning methods across broad domains where covariance matrices emerge. Notably, we elucidate the conceptual insights facilitated by VNNs to the specific task of characterizing brain age gap for neurodegenerative conditions using neuroimaging datasets, a timely problem in computational neuroscience. Broader impacts to other application domains are discussed as well.