Swift-SVD: Theoretical Optimality Meets Practical Efficiency in Low-Rank LLM Compression
On the map: Efficient LLM inference and serving → KV cache optimization
Abstract (arXiv)
The deployment of Large Language Models is constrained by the memory and bandwidth demands of static weights and dynamic Key-Value cache. SVD-based compression provides a hardware-friendly solution to reduce these costs. However, existing methods suffer from two key limitations: some are suboptimal in reconstruction error, while others are theoretically optimal but practically inefficient. In this paper, we propose Swift-SVD, an activation-aware, closed-form compression framework that simultaneously guarantees theoretical optimum, practical efficiency and numerical stability. Swift-SVD incrementally aggregates covariance of output activations given a batch of inputs and performs a single eigenvalue decomposition after aggregation, enabling training-free, fast, and optimal layer-wise low-rank approximation. We employ effective rank to analyze local layer-wise compressibility and design a dynamic rank allocation strategy that jointly accounts for local reconstruction loss and end-to-end layer importance. Extensive experiments across six LLMs and eight datasets demonstrate that Swift-SVD outperforms state-of-the-art baselines, achieving optimal compression accuracy while delivering 3-70X speedups in end-to-end compression time. Our code is available at https://github.com/hiahei/Swift-SVD.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in KV cache optimization, one of 12 directions in Efficient LLM inference and serving. Builds on 35 of its 51 references that are in the map. First posted on arXiv 2 Apr 2026.
Builds on
35 of 51 references.
Closest ancestors · 3
Same direction, newest first
- 2026KV-CoRE: Benchmarking Data-Dependent Low-Rank Compressibility of KV-Caches in LLMs5 Feb
- 2025Towards Economical Inference: Enabling DeepSeek's Multi-Head Latent Attention in Any Transformer-based LLMs20 Feb
- 2024Keep the Cost Down: A Review on Methods to Optimize LLM' s KV-Cache Consumption25 Jul
Other references · 32
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
- 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2016Pointer Sentinel Mixture Modelscited by 2,992
- 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
- 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
- 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
- 2019MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalismscited by 653
- 2021Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depthcited by 411
- 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 378
20 more
- 2024ShortGPT: Layers in Large Language Models are More Redundant Than You Expectcited by 369
- 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 231
- 2024A Survey on Efficient Inference for Large Language Modelscited by 197
- 2022Language model compression with weighted low-rank factorizationcited by 192
- 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 156
- 2022Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapsecited by 136
- 2021Scatterbrain: Unifying Sparse and Low-rank Attention Approximationcited by 128
- 2023LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximationcited by 103
- 2020Low-Rank Bottleneck in Multi-head Attention Modelscited by 88
- 2024Loki: Low-rank Keys for Efficient Sparse Attentioncited by 87
- 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 63
- 2023Transformers learn through gradual rank increasecited by 52
- 2024MoDeGPT: Modular Decomposition for Large Language Model Compressioncited by 45
- 2025Dobi-SVD: Differentiable SVD for LLM Compression and Some New Perspectivescited by 45
- 2024Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compressioncited by 34
- 2024Compressible Dynamics in Deep Overparameterized Low-Rank Learning & Adaptationcited by 26
- 2024Understanding Layer Significance in LLM Alignmentcited by 19
- 2025SlimLLM: Accurate Structured Pruning for Large Language Modelscited by 18
- 2025HPU: High-Bandwidth Processing Unit for Scalable, Cost-effective LLM Inference via GPU Co-processingcited by 5
- 2025Low-Rank Compression of Language Models via Differentiable Rank Selectioncited by 2
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026IO-SVD: Input-Output Whitened SVD for Adaptive-Rank LLM Compression
- 2025Activation-Informed Pareto-Guided Low-Rank Compression for Efficient LLM/VLM
- 2025AdaSVD: Adaptive Singular Value Decomposition for Large Language Models
- 2025FLRC: Fine-grained Low-Rank Compressor for Efficient LLM Inference
- 2026Beyond Variance: Knowledge-Aware LLM Compression via Fisher-Aligned Subspace Diagnostics
- 2023Compressing LLMs: The Truth is Rarely Pure and Never Simple
- 2025FLAT-LLM: Fine-grained Low-rank Activation Space Transformation for Large Language Model Compression
- 2023Rethinking Compression: Reduced Order Modelling of Latent Features in Large Language Models