# Globally optimized SVD compression of LLMs via Fermi-function-based rank selection and gauge fixing

arXiv: https://arxiv.org/abs/2512.03062 · HTML: https://mlsanity.com/paper/2512.03062/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 10 references that are in the map.
- First posted on arXiv 26 Nov 2025.

## Builds on (most cited first, 8 of 8)

- [LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale](https://mlsanity.com/paper/2208.07339/) (2022)
- [SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression](https://mlsanity.com/paper/2403.07378/) (2024)
- [The Unreasonable Ineffectiveness of the Deeper Layers](https://mlsanity.com/paper/2403.17887/) (2024)
- [ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Models](https://mlsanity.com/paper/2312.05821/) (2023)
- [Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectives](https://mlsanity.com/paper/2502.02723/) (2025)
- [LatentLLM: Attention-Aware Joint Tensor Compression](https://mlsanity.com/paper/2505.18413/) (2025)
- [COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation](https://mlsanity.com/paper/2507.07580/) (2025)
- [Pivoting Factorization: A Compact Meta Low-Rank Representation of Sparsity for Efficient Inference in Large Language Models](https://mlsanity.com/paper/2501.19090/) (2025)

## Built on by (most cited first, 3 of 3)

- [AA-SVD : Anchored and Adaptive SVD for Large Language Model Compression](https://mlsanity.com/paper/2604.02119/) (2026)
- [LLM Compression by Block Removal with Constrained Binary Optimization](https://mlsanity.com/paper/2602.00161/) (2026)
- [Weight Clustering Reveals the Importance of Relative Rank in Large Language Models](https://mlsanity.com/paper/2603.17917/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are very demanding in terms of their computational resources. Low-rank decompositions of LLM weights, e.g. via Singular Value Decomposition (SVD), is a promising approach for LLM compression, but presents several practical hurdles, e.g. selecting appropriate layer-wise ranks and getting rid of its parameter redundancy. In this work, we present two physics-inspired improvements to SVD LLM compression: (1) FermiGrad, a gradient-descent algorithm that determines globally optimal layer-wise ranks by relaxing the discrete singular-value truncation into a continuous optimization using the Fermi function; (2) PivGa, an additional textitlossless compression of the low-rank factors that exploits the intrinsic gauge freedom in their parametrization.