mlsanityData to 14 Oct 2026

Thanos: A Block-wise Pruning Algorithm for Efficient Large Language Model Compression

arXiv 2504.05346 ↗First posted 6 Apr 2025cs.LG cs.AI cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper presents Thanos, a novel weight-pruning algorithm designed to reduce the memory footprint and enhance the computational efficiency of large language models (LLMs) by removing redundant weights while maintaining accuracy. Thanos introduces a block-wise pruning strategy with adaptive masks that dynamically adjust to weight importance, enabling flexible sparsity patterns and structured formats, such as n:m sparsity, optimized for hardware acceleration. Experimental evaluations demonstrate that Thanos achieves state-of-the-art performance in structured pruning and outperforms existing methods in unstructured pruning. By providing an efficient and adaptable approach to model compression, Thanos offers a practical solution for deploying large models in resource-constrained environments.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 39 of its 53 references that are in the map. First posted on arXiv 6 Apr 2025.

Builds on

39 of 53 references.

References · 39

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2024The Llama 3 Herd of Modelscited by 14,715
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  8. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,100
  9. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  10. 2016Pointer Sentinel Mixture Modelscited by 3,096
  11. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  12. 2022Emergent Abilities of Large Language Modelscited by 2,512
27 more
  1. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  2. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,183
  3. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,164
  4. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  5. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  6. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  7. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,121
  8. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  9. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  10. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  11. 2017To prune, or not to prune: exploring the efficacy of pruning for model compressioncited by 603
  12. 2020What is the State of Neural Network Pruning?cited by 583
  13. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 575
  14. 2019Rigging the Lottery: Making All Tickets Winnerscited by 495
  15. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  16. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  17. 2022Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruningcited by 322
  18. 2024Massive Activations in Large Language Modelscited by 294
  19. 2021Learning N:M Fine-grained Structured Sparse Neural Networks From Scratchcited by 179
  20. 2022A Fast Post-Training Pruning Framework for Transformerscited by 162
  21. 2023Quantizable Transformers: Removing Outliers by Helping Attention Heads Do Nothingcited by 151
  22. 2021Group Fisher Pruning for Practical Network Compressioncited by 131
  23. 2021Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Maskscited by 103
  24. 2024PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compressioncited by 61
  25. 2021AC/DC: Alternating Compressed/DeCompressed Training of Deep Neural Networkscited by 59
  26. 2024The Super Weight in Large Language Modelscited by 52
  27. 2023Intriguing Properties of Quantization at Scalecited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025PATCH: Learnable Tile-level Hybrid Sparsity for LLMscited by 3
  2. 2025Think Before You Prune: Selective Self-Generated Calibration for Pruning Large Reasoning Modelscited by 2
  3. 2025OPTIMA: Optimal One-shot Pruning for LLMs via Quadratic Programming Reconstructioncited by 2
  4. 2025StructPrune: Structured Global Pruning asymptotics with O(√N) GPU Memorycited by 1
  5. 2026LEAP: Learnable End-to-End Adaptive Pruning of Large Language Modelscited by 0
  6. 2026Compression Trinity: Exploring Sparsity, Quantization, and Low-Rank Approximations for LLM Compressioncited by 0
  7. 2026QK-Wanda: Coupling Queries and Keys for Unstructured Pruningcited by 0