Fast optimization of common basis for matrix set through Common Singular Value Decomposition

SVD (singular value decomposition) is one of the basic tools of machine learning, allowing to optimize basis for a given matrix. However, sometimes we have a set of matrices Akk instead, and would like to optimize a single common basis for them: find orthogonal matrices U, V, such that UT Ak V set of matrices is somehow simpler. For example DCT-II is orthonormal basis of functions commonly used in image/video compression - as discussed here, this kind of basis can be quickly automatically optimized for a given dataset. While also discussed gradient descent optimization might be computationally costly, there is proposed CSVD (common SVD): fast general approach based on SVD. Specifically, we choose U as built of eigenvectors of sumi (wk)q (Ak AkT)p and V of sumk (wk)q (AkT Ak)p, where wk are their weights, p,q>0 are some chosen powers e.g. 1/2, optionally with normalization e.g. A → A - rcT where ri=sumj Aij, cj =sumi Aij.