# The Singular Value Decomposition, Applications and Beyond

arXiv: https://arxiv.org/abs/1510.08532 · HTML: https://mlsanity.com/paper/1510.08532/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Oct 2015.

## Built on by (most cited first, 11 of 11)

- [On Predictability of Reinforcement Learning Dynamics for Large Language Models](https://mlsanity.com/paper/2510.00553/) (2025)
- [Learning to Foresee: Unveiling the Unlocking Efficiency of On-Policy Distillation](https://mlsanity.com/paper/2605.11739/) (2026)
- [Pay Attention to Small Weights](https://mlsanity.com/paper/2506.21374/) (2025)
- [Quantum-inspired Techniques in Tensor Networks for Industrial Contexts](https://mlsanity.com/paper/2404.11277/) (2024)
- [Large Language Model Evaluation via Matrix Nuclear-Norm](https://mlsanity.com/paper/2410.10672/) (2024)
- [AICD Bench: A Challenging Benchmark for AI-Generated Code Detection](https://mlsanity.com/paper/2602.02079/) (2026)
- [Gradient Clipping Beyond Vector Norms: A Spectral Approach for Matrix-Valued Parameters](https://mlsanity.com/paper/2605.11838/) (2026)
- [AC-LoRA: Auto Component LoRA for Personalized Artistic Style Image Generation](https://mlsanity.com/paper/2504.02231/) (2025)
- [T'ecnicas Quantum-Inspired en Tensor Networks para Contextos Industriales](https://mlsanity.com/paper/2404.17645/) (2024)
- [Eliot: Interactively underlineExploring Fast-Changing Scientific underlineLiterature Trends with underlineOnline Daunderlineta and Learning](https://mlsanity.com/paper/2605.27610/) (2026)
- [MAAT: Multi-phase Adapter-Aware Targeted Unlearning](https://mlsanity.com/paper/2605.30514/) (2026)

## Abstract (arXiv)

The singular value decomposition (SVD) is not only a classical theory in matrix computation and analysis, but also is a powerful tool in machine learning and modern data analysis. In this tutorial we first study the basic notion of SVD and then show the central role of SVD in matrices. Using majorization theory, we consider variational principles of singular values and eigenvalues. Built on SVD and a theory of symmetric gauge functions, we discuss unitarily invariant norms, which are then used to formulate general results for matrix low rank approximation. We study the subdifferentials of unitarily invariant norms. These results would be potentially useful in many machine learning problems such as matrix completion and matrix data classification. Finally, we discuss matrix low rank approximation and its recent developments such as randomized SVD, approximate matrix multiplication, CUR decomposition, and Nystrom approximation. Randomized algorithms are important approaches to large scale SVD as well as fast matrix computations.