mlsanityData to 14 Oct 2026

Accelerating Attention with Basis Decomposition

arXiv 2510.01718 ↗First posted 2 Oct 2025cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Attention is a core operation in large language models (LLMs). We present BD Attention (BDA), a lossless algorithmic reformulation of attention. BDA is enabled by a simple matrix identity from Basis Decomposition (BD), which restructures multi-head projections into a compact form while preserving exact outputs. Unlike I/O-aware system optimizations such as FlashAttention, BDA provides a mathematically guaranteed acceleration that is architecture-agnostic. On DeepSeek-V2-Lite (16B, FP16), BDA requires only 4s of offline preparation with no retraining required and, on modern GPUs, achieves 34% faster key/value projections and 25% smaller weights, while increasing perplexity (PPL) by just 0.02% (FP16) or 0.0004% (FP32), a negligible effect on model performance. These results position BDA as a theoretically exact method for lossless attention acceleration that is complementary to existing engineering-level optimizations. Our code is available at https://github.com/abcbdf/basis-decomposition-official.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 60 of its 72 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

60 of 72 references.

References · 60

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  7. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  9. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  11. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
38 more
  1. 2020Longformer: The Long-Document Transformercited by 3,413
  2. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  3. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  4. 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
  5. 2020Reformer: The Efficient Transformercited by 1,668
  6. 2020Rethinking Attention with Performerscited by 1,601
  7. 2019Generating Long Sequences with Sparse Transformerscited by 1,546
  8. 2020Linformer: Self-Attention with Linear Complexitycited by 1,474
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  10. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  11. 2024DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Modelcited by 1,135
  12. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  13. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  14. 2024DoRA: Weight-Decomposed Low-Rank Adaptationcited by 758
  15. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  16. 2024FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precisioncited by 519
  17. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 502
  18. 2024SliceGPT: Compress Large Language Models by Deleting Rows and Columnscited by 385
  19. 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 300
  20. 2024PiSSA: Principal Singular Values and Singular Vectors Adaptation of Large Language Modelscited by 269
  21. 2024SVD-LLM: Truncation-aware Singular Value Decomposition for Large Language Model Compressioncited by 243
  22. 2021Accelerating Sparse Deep Neural Networkscited by 223
  23. 2023ReLoRA: High-Rank Training Through Low-Rank Updatescited by 204
  24. 2022Language model compression with weighted low-rank factorizationcited by 195
  25. 2023ASVD: Activation-aware Singular Value Decomposition for Compressing Large Language Modelscited by 169
  26. 2024DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMscited by 167
  27. 2020Towards Theoretically Understanding Why SGD Generalizes Better Than ADAM in Deep Learningcited by 126
  28. 2023The Truth is in There: Improving Reasoning in Language Models with Layer-Selective Rank Reductioncited by 123
  29. 2023LoSparse: Structured Compression of Large Language Models based on Low-Rank and Sparse Approximationcited by 104
  30. 2021Skyformer: Remodel Self-Attention with Gaussian Kernel and Nystr"om Methodcited by 63
  31. 2024Compressing Large Language Models using Low Rank and Low Precision Decompositioncited by 59
  32. 2021Initialization and Regularization of Factorized Neural Layerscited by 58
  33. 2024MoDeGPT: Modular Decomposition for Large Language Model Compressioncited by 48
  34. 2023IncreLoRA: Incremental Parameter Allocation Method for Parameter-Efficient Fine-tuningcited by 45
  35. 2024SLTrain: a sparse plus low-rank approach for parameter and memory efficient pretrainingcited by 45
  36. 2022Low-rank lottery tickets: finding efficient low-rank neural networks via matrix differential equationscited by 43
  37. 2023The LLM Surgeoncited by 38
  38. 2022Exploring Low Rank Training of Deep Neural Networkscited by 30

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.