mlsanityData to 14 Oct 2026

Optimizing LLMs for Resource-Constrained Environments: A Survey of Model Compression Techniques

arXiv 2505.02309 ↗First posted 5 May 2025cs.LG cs.AI cs.CLcited by 9

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have revolutionized many areas of artificial intelligence (AI), but their substantial resource requirements limit their deployment on mobile and edge devices. This survey paper provides a comprehensive overview of techniques for compressing LLMs to enable efficient inference in resource-constrained environments. We examine three primary approaches: Knowledge Distillation, Model Quantization, and Model Pruning. For each technique, we discuss the underlying principles, present different variants, and provide examples of successful applications. We also briefly discuss complementary techniques such as mixture-of-experts and early-exit strategies. Finally, we highlight promising future directions, aiming to provide a valuable resource for both researchers and practitioners seeking to optimize LLMs for edge deployment.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 50 of its 62 references that are in the map. First posted on arXiv 5 May 2025.

Builds on

50 of 62 references.

References · 50

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  5. 2020Retrieval-Augmented Generation for Knowledge-Intensive NLP Taskscited by 9,297
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  7. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  8. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  9. 2018The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networkscited by 2,262
  10. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  12. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
38 more
  1. 2014FitNets: Hints for Thin Deep Netscited by 1,653
  2. 2020Knowledge Distillation: A Surveycited by 1,603
  3. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  4. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  5. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  6. 2022LaMDA: Language Models for Dialog Applicationscited by 1,220
  7. 2018Hierarchical Neural Story Generationcited by 1,193
  8. 2023On-Policy Distillation of Language Models: Learning from Self-Generated Mistakescited by 939
  9. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  10. 2019Relational Knowledge Distillationcited by 889
  11. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  12. 2023Distilling Step-by-Step! Outperforming Larger Language Models with Less Training Data and Smaller Model Sizescited by 737
  13. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 621
  14. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  15. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  16. 2018Born Again Neural Networkscited by 546
  17. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  18. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 521
  19. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  20. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 479
  21. 2013Do Deep Nets Really Need to be Deep?cited by 474
  22. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  23. 2020Movement Pruning: Adaptive Sparsity by Fine-Tuningcited by 403
  24. 2018AMC: AutoML for Model Compression and Acceleration on Mobile Devicescited by 378
  25. 2022Large Language Models Are Reasoning Teacherscited by 329
  26. 2018Model compression via distillation and quantizationcited by 289
  27. 2023A Survey on Model Compression for Large Language Modelscited by 269
  28. 2021Pruning and Quantization for Deep Neural Network Acceleration: A Surveycited by 263
  29. 2016Ternary Weight Networkscited by 204
  30. 2022FP8 Formats for Deep Learningcited by 196
  31. 2023BitNet: Scaling 1-bit Transformers for Large Language Modelscited by 189
  32. 2024DistiLLM: Towards Streamlined Distillation for Large Language Modelscited by 163
  33. 2015Structured Pruning of Deep Convolutional Neural Networkscited by 149
  34. 2021Understanding and Overcoming the Challenges of Efficient Transformer Quantizationcited by 141
  35. 2023ZeroQuant-V2: Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensationcited by 79
  36. 2024A Survey on Transformer Compressioncited by 46
  37. 2022Learning Best Combination for Efficient N:M Sparsitycited by 32
  38. 2024LLaMA-NAS: Efficient Neural Architecture Search for Large Language Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 5; 2026: 4.

9 papers.

  1. 2025Few-Shot Knowledge Distillation of LLMs With Counterfactual Explanationscited by 4
  2. 2025Quantized Large Language Models in Biomedical Natural Language Processing: Evaluation and Recommendationcited by 3
  3. 2025From Construction to Injection: Edit-Based Fingerprints for Large Language Modelscited by 2
  4. 2025RCPU: Rotation-Constrained Error Compensation for Structured Pruning of Large Language Modelscited by 2
  5. 2025Post-Training Quantization of OpenPangu Models for Efficient Deployment on Atlas A2cited by 1
  6. 2026HyFunc: Accelerating LLM-based Function Calls for Agentic AI through Hybrid-Model Cascade and Dynamic Templatingcited by 1
  7. 2026Physical Foundation Models: Fixed hardware implementations of large-scale neural networkscited by 1
  8. 2026Risk-Conditioned Fine-Tuning of Large Language Modelscited by 1
  9. 2026Do Student LLMs Inherit OOD Robustness? Invariance-Weighted Distillation for Reliable Knowledge Transfercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insights
  2. 2024A Survey on Efficient Inference for Large Language Models
  3. 2025Taming the Titans: A Survey of Efficient LLM Inference Serving
  4. 2024On the Compressibility of Quantized Large Language Models
  5. 2024Edge Intelligence Optimization for Large Language Model Inference with Batching and Quantization
  6. 2025TOGGLE: Temporal Logic-Guided Large Language Model Compression for Edge
  7. 2025From Large to Super-Tiny: End-to-End Optimization for Cost-Efficient LLMs
  8. 2024Inference Optimizations for Large Language Models: Effects, Challenges, and Practical Considerations