# Backpropagation-Friendly Eigendecomposition

arXiv: https://arxiv.org/abs/1906.09023 · HTML: https://mlsanity.com/paper/1906.09023/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Jun 2019.

## Built on by (most cited first, 27 of 27)

- [SPARF: Neural Radiance Fields from Sparse and Noisy Poses](https://mlsanity.com/paper/2211.11738/) (2022)
- [No Pain, Big Gain: Classify Dynamic Point Cloud Sequences with Static Models by Fitting Feature-level Space-time Surfaces](https://mlsanity.com/paper/2203.11113/) (2022)
- [Why Approximate Matrix Square Root Outperforms Accurate SVD in Global Covariance Pooling?](https://mlsanity.com/paper/2105.02498/) (2021)
- [Equivariant Manifold Flows](https://mlsanity.com/paper/2107.08596/) (2021)
- [Scribble-Supervised Semantic Segmentation by Uncertainty Reduction on Neural Representation and Self-Supervision on Neural Eigenspace](https://mlsanity.com/paper/2102.09896/) (2021)
- [Robust Differentiable SVD](https://mlsanity.com/paper/2104.03821/) (2021)
- [Fast Differentiable Matrix Square Root and Inverse Square Root](https://mlsanity.com/paper/2201.12543/) (2022)
- [Fast Differentiable Matrix Square Root](https://mlsanity.com/paper/2201.08663/) (2022)
- [MatchNorm: Learning-based Point Cloud Registration for 6D Object Pose Estimation in the Real World](https://mlsanity.com/paper/2203.15309/) (2022)
- [MG-GNN: Multigrid Graph Neural Networks for Learning Multilevel Domain Decomposition Methods](https://mlsanity.com/paper/2301.11378/) (2023)
- [Self-Consistency Training for Density-Functional-Theory Hamiltonian Prediction](https://mlsanity.com/paper/2403.09560/) (2024)
- [Learning Interface Conditions in Domain Decomposition Solvers](https://mlsanity.com/paper/2205.09833/) (2022)
- [Stochastic Whitening Batch Normalization](https://mlsanity.com/paper/2106.04413/) (2021)
- [3D Generative Model Latent Disentanglement via Local Eigenprojection](https://mlsanity.com/paper/2302.12798/) (2023)
- [Graph Neural Networks and Applied Linear Algebra](https://mlsanity.com/paper/2310.14084/) (2023)
- [Grouping-matrix based Graph Pooling with Adaptive Number of Clusters](https://mlsanity.com/paper/2209.02939/) (2022)
- [Batch-efficient EigenDecomposition for Small and Medium Matrices](https://mlsanity.com/paper/2207.04228/) (2022)
- [Learning truly monotone operators with applications to nonlinear inverse problems](https://mlsanity.com/paper/2404.00390/) (2024)
- [Aspect Based Sentiment Analysis Using Spectral Temporal Graph Neural Network](https://mlsanity.com/paper/2202.06776/) (2022)
- [Gaussian-Mixture-Model Q-Functions for Policy Iteration in Reinforcement Learning](https://mlsanity.com/paper/2512.18763/) (2025)
- [Projected Sliced Wasserstein Autoencoder-based Hyperspectral Images Anomaly Detection](https://mlsanity.com/paper/2112.11243/) (2021)
- [Interpretable neural architecture search and transfer learning for understanding CRISPR/Cas9 off-target enzymatic reactions](https://mlsanity.com/paper/2305.11917/) (2023)
- [Informed Decision-Making through Advancements in Open Set Recognition and Unknown Sample Detection](https://mlsanity.com/paper/2405.05836/) (2024)
- [AI-enhanced tuning of quantum dot Hamiltonians toward Majorana modes](https://mlsanity.com/paper/2601.02149/) (2026)
- [NeuraLSP: A Neural Spectral Preconditioner for Accelerating PDE Solvers](https://mlsanity.com/paper/2601.20174/) (2026)
- [Incremental Graph Construction Enables Robust Spectral Clustering of Texts](https://mlsanity.com/paper/2603.03056/) (2026)
- [Voronoi-based Second-order Descriptor with Whitened Metric in LiDAR Place Recognition](https://mlsanity.com/paper/2603.14974/) (2026)

## Abstract (arXiv)

Eigendecomposition (ED) is widely used in deep networks. However, the backpropagation of its results tends to be numerically unstable, whether using ED directly or approximating it with the Power Iteration method, particularly when dealing with large matrices. While this can be mitigated by partitioning the data in small and arbitrary groups, doing so has no theoretical basis and makes its impossible to exploit the power of ED to the full. In this paper, we introduce a numerically stable and differentiable approach to leveraging eigenvectors in deep networks. It can handle large matrices without requiring to split them. We demonstrate the better robustness of our approach over standard ED and PI for ZCA whitening, an alternative to batch normalization, and for PCA denoising, which we introduce as a new normalization strategy for deep networks, aiming to further denoise the network's features.