# Fast optimization of common basis for matrix set through Common Singular Value Decomposition

arXiv: https://arxiv.org/abs/2204.08242 · HTML: https://mlsanity.com/paper/2204.08242/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 0 of its 8 references that are in the map.
- First posted on arXiv 18 Apr 2022.

## Built on by (most cited first, 2 of 2)

- [Biology-inspired joint distribution neurons based on Hierarchical Correlation Reconstruction allowing for multidirectional propagation of values and densities](https://mlsanity.com/paper/2405.05097/) (2024)
- [Linear cost mutual information estimation and independence test of similar performance as HSIC](https://mlsanity.com/paper/2508.18338/) (2025)

## Abstract (arXiv)

SVD (singular value decomposition) is one of the basic tools of machine learning, allowing to optimize basis for a given matrix. However, sometimes we have a set of matrices Akk instead, and would like to optimize a single common basis for them: find orthogonal matrices U, V, such that UT Ak V set of matrices is somehow simpler. For example DCT-II is orthonormal basis of functions commonly used in image/video compression - as discussed here, this kind of basis can be quickly automatically optimized for a given dataset. While also discussed gradient descent optimization might be computationally costly, there is proposed CSVD (common SVD): fast general approach based on SVD. Specifically, we choose U as built of eigenvectors of sumi (wk)q (Ak AkT)p and V of sumk (wk)q (AkT Ak)p, where wk are their weights, p,q>0 are some chosen powers e.g. 1/2, optionally with normalization e.g. A → A - rcT where ri=sumj Aij, cj =sumi Aij.