mlsanityData to 7 Oct 2026

Beyond Efficiency: A Systematic Survey of Resource-Efficient Large Language Models

arXiv 2401.00625 ↗First posted 1 Jan 2024cs.LGcited by 64

Not on the map. Why?

Abstract (arXiv)

The burgeoning field of Large Language Models (LLMs), exemplified by sophisticated models like OpenAI's ChatGPT, represents a significant advancement in artificial intelligence. These models, however, bring forth substantial challenges in the high consumption of computational, memory, energy, and financial resources, especially in environments with limited resource capabilities. This survey aims to systematically address these challenges by reviewing a broad spectrum of techniques designed to enhance the resource efficiency of LLMs. We categorize methods based on their optimization focus: computational, memory, energy, financial, and network resources and their applicability across various stages of an LLM's lifecycle, including architecture design, pretraining, finetuning, and system design. Additionally, the survey introduces a nuanced categorization of resource efficiency techniques by their specific resource types, which uncovers the intricate relationships and mappings between various resources and corresponding optimization techniques. A standardized set of evaluation metrics and datasets is also presented to facilitate consistent and fair comparisons across different models and techniques. By offering a comprehensive overview of the current sota and identifying open research avenues, this survey serves as a foundational reference for researchers and practitioners, aiding them in developing more sustainable and efficient LLMs in a rapidly evolving landscape.

Key facts. Cited by 64 papers in the map (data through 7 Oct 2026). First posted on arXiv 1 Jan 2024.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 29; 2025: 19; 2026: 16.

64 papers.

Sort:

By area:

  1. 2024A Survey on the Memory Mechanism of Large Language Model based Agentscited by 622
  2. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  3. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  4. 2024A Survey on Inference Optimization Techniques for Mixture of Experts Modelscited by 46
  5. 2024Uncertainty Quantification for In-Context Learning of Large Language Modelscited by 45
  6. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  7. 2024R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?cited by 29
  8. 2024Fast yet Safe: Early-Exiting with Risk Controlcited by 27
  9. 2024SparseLLM: Towards Global Pruning for Pre-trained Language Modelscited by 23
  10. 2024MVGamba: Unify 3D Content Generation as State Space Sequence Modelingcited by 23
  11. 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
  12. 2025The Rise of Small Language Models in Healthcare: A Comprehensive Surveycited by 18
38 more
  1. 2024Abstractive Text Summarization: State of the Art, Challenges, and Improvementscited by 17
  2. 2024Energy-Latency Manipulation of Multi-modal Large Language Models via Verbose Samplescited by 14
  3. 2024Pruning as a Domain-specific LLM Extractorcited by 12
  4. 2025LLM-Oriented Token-Adaptive Knowledge Distillationcited by 12
  5. 2024ELAD: Explanation-Guided Large Language Models Active Distillationcited by 11
  6. 2024Persistent Topological Features in Large Language Modelscited by 11
  7. 2025RARE: Retrieval-Augmented Reasoning Modelingcited by 11
  8. 2025AudioGen-Omni: A Unified Multimodal Diffusion Transformer for Video-Synchronized Audio, Speech, and Song Generationcited by 11
  9. 2024DriveGenVLM: Real-world Video Generation for Vision Language Model based Autonomous Drivingcited by 10
  10. 2024Advancing Object Detection in Transportation with Multimodal Large Language Models (MLLMs): A Comprehensive Review and Empirical Testingcited by 10
  11. 2025Neural Brain: A Neuroscience-inspired Framework for Embodied Agentscited by 10
  12. 2024Can LLMs Understand Social Norms in Autonomous Driving Games?cited by 6
  13. 2025LatentLLM: Attention-Aware Joint Tensor Compressioncited by 6
  14. 2025Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniquescited by 5
  15. 2026Watt Counts: Energy-Aware Benchmark for Sustainable LLM Inference on Heterogeneous GPU Architecturescited by 5
  16. 2024Achieving Peak Performance for Large Language Models: A Systematic Reviewcited by 4
  17. 2024From Linguistic Giants to Sensory Maestros: A Survey on Cross-Modal Reasoning with Large Language Modelscited by 4
  18. 2024Dynamic Vocabulary Pruning in Early-Exit LLMscited by 4
  19. 2024Explaining latent representations of generative models with large multimodal modelscited by 3
  20. 2025Green Prompting: Characterizing Prompt-driven Energy Costs of LLM Inferencecited by 3
  21. 2025μ-MoE: Test-Time Pruning as Micro-Grained Mixture-of-Expertscited by 3
  22. 2026A Survey on the Green Development of Large Models: From Resource-Efficient Architectures to Hardware-Software Co-Designcited by 3
  23. 2024A Condensed Transition Graph Framework for Zero-shot Link Prediction with Large Language Modelscited by 2
  24. 2024On-Device LLMs for SMEs: Challenges and Opportunitiescited by 2
  25. 2025FISCAL: Financial Synthetic Claim-document Augmented Learning for Efficient Fact-Checkingcited by 2
  26. 2026Revisiting the Effectiveness of LLM Pruning for Test-Time Scalingcited by 2
  27. 2024LatentExplainer: Explaining Latent Representations in Deep Generative Models with Multimodal Large Language Modelscited by 1
  28. 2025Cross-modal RAG: Sub-dimensional Text-to-Image Retrieval-Augmented Generationcited by 1
  29. 2025Comparing energy consumption and accuracy in text classification inferencecited by 1
  30. 2025StructPrune: Structured Global Pruning asymptotics with O(√N) GPU Memorycited by 1
  31. 2025Integrating Machine Learning into Belief-Desire-Intention Agents: Current Advances and Open Challengescited by 1
  32. 2026Prompt Optimization Via Diffusion Language Modelscited by 1
  33. 2026An Exploration-Analysis-Disambiguation Reasoning Framework for Word Sense Disambiguation with Low-Parameter LLMscited by 1
  34. 2026TTQ: Activation-Aware Test-Time Quantization to Accelerate LLM Inference On The Flycited by 1
  35. 2026DrugSAGE:Self-evolving Agent Experience for Efficient State-of-the-Art Drug Discoverycited by 1
  36. 2024OpenBA-V2: Reaching 77.3% High Compression Ratio with Fast Multi-Stage Pruningcited by 0
  37. 2024Enhancing Security Control Production With Generative AIcited by 0
  38. 2024Software Performance Engineering for Foundation Model-Powered Softwarecited by 0

50 most cited of 64 All 64 (JSON)