# SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compression

arXiv: https://arxiv.org/abs/2503.12340 · HTML: https://mlsanity.com/paper/2503.12340/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 34 references that are in the map.
- First posted on arXiv 16 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [A Survey of Large Language Models](https://mlsanity.com/paper/2303.18223/) (2023)
- [Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering](https://mlsanity.com/paper/1809.02789/) (2018)
- [GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers](https://mlsanity.com/paper/2210.17323/) (2022)
- [SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://mlsanity.com/paper/2301.00774/) (2023)
- [MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms](https://mlsanity.com/paper/1905.13319/) (2019)
- [SliceGPT: Compress Large Language Models by Deleting Rows and Columns](https://mlsanity.com/paper/2401.15024/) (2024)
- [SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compression](https://mlsanity.com/paper/2403.07378/) (2024)
- [Language model compression with weighted low-rank factorization](https://mlsanity.com/paper/2207.00112/) (2022)
- [BiLLM: Pushing the Limit of Post-Training Quantization for LLMs](https://mlsanity.com/paper/2402.04291/) (2024)
- [What Matters in Transformers? Not All Attention is Needed](https://mlsanity.com/paper/2406.15786/) (2024)
- [MoDeGPT: Modular Decomposition for Large Language Model Compression](https://mlsanity.com/paper/2408.09632/) (2024)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inference](https://mlsanity.com/paper/2502.17599/) (2025)
- [An Automated Survey of Generative Artificial Intelligence: Large Language Models, Architectures, Protocols, and Applications](https://mlsanity.com/paper/2306.02781/) (2023)
- [The Internet of Things in the Era of Generative AI: Vision and Challenges](https://mlsanity.com/paper/2401.01923/) (2024)
- [Greedy Output Approximation: Towards Efficient Structured Pruning for LLMs Without Retraining](https://mlsanity.com/paper/2407.19126/) (2024)
- [Famba-V: Fast Vision Mamba with Cross-Layer Token Fusion](https://mlsanity.com/paper/2409.09808/) (2024)
- [ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization](https://mlsanity.com/paper/2411.07762/) (2024)

## Built on by (most cited first, 30 of 67)

- [AI Flow: Perspectives, Scenarios, and Approaches](https://mlsanity.com/paper/2506.12479/) (2025)
- [Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression](https://mlsanity.com/paper/2505.19433/) (2025)
- [QSVD: Efficient Low-rank Approximation for Unified Query-Key-Value Weight Compression in Low-Precision Vision-Language Models](https://mlsanity.com/paper/2510.16292/) (2025)
- [SAES-SVD: Self-Adaptive Suppression of Accumulated and Local Errors for SVD-based LLM Compression](https://mlsanity.com/paper/2602.03051/) (2026)
- [LittleBit: Ultra Low-Bit Quantization via Latent Factorization](https://mlsanity.com/paper/2506.13771/) (2025)
- [FlashSVD: Memory-Efficient Inference with Streaming for Low-Rank Models](https://mlsanity.com/paper/2508.01506/) (2025)
- [OSCAR: Offline Spectral Covariance-Aware Rotation for 2-bit KV Cache Quantization](https://mlsanity.com/paper/2605.17757/) (2026)
- [A3 : an Analytical Low-Rank Approximation Framework for Attention](https://mlsanity.com/paper/2505.12942/) (2025)
- [Zero Sum SVD: Balancing Loss Sensitivity for Low Rank LLM Compression](https://mlsanity.com/paper/2602.02848/) (2026)
- [Model Merging in the Essential Subspace](https://mlsanity.com/paper/2602.20208/) (2026)
- [AA-SVD : Anchored and Adaptive SVD for Large Language Model Compression](https://mlsanity.com/paper/2604.02119/) (2026)
- [CoSpaDi: Compressing LLMs via Calibration-Guided Sparse Dictionary Learning](https://mlsanity.com/paper/2509.22075/) (2025)
- [UniQL: Unified Quantization and Low-rank Compression for Adaptive Edge LLMs](https://mlsanity.com/paper/2512.03383/) (2025)
- [Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTS](https://mlsanity.com/paper/2507.05557/) (2025)
- [COALA: Numerically Stable and Efficient Framework for Context-Aware Low-Rank Approximation](https://mlsanity.com/paper/2507.07580/) (2025)
- [MASQuant: Modality-Aware Smoothing Quantization for Multimodal Large Language Models](https://mlsanity.com/paper/2603.04800/) (2026)
- [CARE: Covariance-Aware and Rank-Enhanced Decomposition for Enabling Multi-Head Latent Attention](https://mlsanity.com/paper/2603.17946/) (2026)
- [Addition is almost all you need: Compressing large language models with double binary factorization](https://mlsanity.com/paper/2505.11076/) (2025)
- [Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM](https://mlsanity.com/paper/2510.05544/) (2025)
- [Towards Reversible Model Merging For Low-rank Weights](https://mlsanity.com/paper/2510.14163/) (2025)
- [WSVD: Weighted Low-Rank Approximation for Fast and Efficient Execution of Low-Precision Vision-Language Models](https://mlsanity.com/paper/2604.02570/) (2026)
- [FlashSVD v1.5: Making Low-Rank Transformers Inference Actually Fast](https://mlsanity.com/paper/2605.08314/) (2026)
- [BALF: Budgeted Activation-Aware Low-Rank Factorization for Fine-Tuning-Free Model Compression](https://mlsanity.com/paper/2509.25136/) (2025)
- [FlexRank: Nested Low-Rank Knowledge Decomposition for Adaptive Model Deployment](https://mlsanity.com/paper/2602.02680/) (2026)
- [Ruyi2 Technical Report](https://mlsanity.com/paper/2602.22543/) (2026)
- [Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression](https://mlsanity.com/paper/2604.01609/) (2026)
- [UniSVQ: 2-bit Unified Scalar-Vector Quantization](https://mlsanity.com/paper/2606.10520/) (2026)
- [S3LoRA: Safe Spectral Sharpness-Guided Pruning in Adaptation of Agent Planner](https://mlsanity.com/paper/2508.15068/) (2025)
- [Neural Weight Compression for Language Models](https://mlsanity.com/paper/2510.11234/) (2025)
- [Neuronal Group Communication for Efficient Neural representation](https://mlsanity.com/paper/2510.16851/) (2025)

## Abstract (arXiv)

Despite significant advancements, the practical deployment of Large Language Models (LLMs) is often hampered by their immense sizes, highlighting the need for effective compression techniques. Singular Value Decomposition (SVD) is a promising LLM compression technique. However, existing SVD-based compression methods fall short in reducing truncation losses, leading to less competitive performance in compressed models. In this work, we introduce SVD-LLM V2, a SVD-based LLM compression method that optimizes singular value truncation in SVD compression with two techniques. First, SVD-LLM V2 proposes to use theoretical truncation loss of weight matrices to assign a unique compression ratio to each weight matrix at different layers to accommodate weight redundancy heterogeneity. Second, SVD-LLM V2 proposes loss-optimized weight truncation to ensure that the truncated singular values result in a lower and more stable truncation loss in practice. We evaluate SVD-LLM V2 on ten datasets and five LLMs at various scales. Our results show SVD-LLM V2 outperforms state-of-the-art SVD-based LLM compression methods. Our code is available at https://github.com/AIoT-MLSys-Lab/SVD-LLM