mlsanityData to 7 Oct 2026

EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Voting

arXiv 2406.15758 ↗First posted 22 Jun 2024cs.LG cs.DCcited by 26

Not on the map. Why?

Abstract (arXiv)

Efficient adaption of large language models (LLMs) on edge devices is essential for applications requiring continuous and privacy-preserving adaptation and inference. However, existing tuning techniques fall short because of the high computation and memory overheads. To this end, we introduce a computation- and memory-efficient LLM tuning framework, called Edge-LLM, to facilitate affordable and effective LLM adaptation on edge devices. Specifically, Edge-LLM features three core components: (1) a layer-wise unified compression (LUC) technique to reduce the computation overhead by generating layer-wise pruning sparsity and quantization bit-width policies, (2) an adaptive layer tuning and voting scheme to reduce the memory overhead by reducing the backpropagation depth, and (3) a complementary hardware scheduling strategy to handle the irregular computation patterns introduced by LUC and adaptive layer tuning, thereby achieving efficient computation and data movements. Extensive experiments demonstrate that Edge-LLM achieves a 2.92x speed up and a 4x memory overhead reduction as compared to vanilla tuning methods with comparable task accuracy. Our code is available at https://github.com/GATECH-EIC/Edge-LLM

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 15 of its 21 references that are in the map. First posted on arXiv 22 Jun 2024.

Builds on

15 of 21 references.

References · 15

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,579
  6. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  7. 2023LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attentioncited by 776
  8. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 591
  9. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  10. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  11. 2022LST: Ladder Side-Tuning for Parameter and Memory Efficient Transfer Learningcited by 228
  12. 2023Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantizationcited by 97
3 more
  1. 2021Understanding Softmax Confidence and Uncertaintycited by 62
  2. 2023Master-ASR: Achieving Multilingual Scalability and Low-Resource Adaptation in ASR with Modular Learningcited by 9
  3. 20212-in-1 Accelerator: Enabling Random Precision Switch for Winning Both Adversarial Robustness and Efficiencycited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 18; 2026: 3.

26 papers.

Sort:

By area:

  1. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  2. 2025Vision-Language Models for Edge Networks: A Comprehensive Surveycited by 29
  3. 2025SLED: A Speculative LLM Decoding Framework for Efficient Edge Servingcited by 13
  4. 2024WDMoE: Wireless Distributed Mixture of Experts for Large Language Modelscited by 12
  5. 2024When IoT Meet LLMs: Applications and Challengescited by 11
  6. 2024Leveraging Large Language Models for Integrated Satellite-Aerial-Terrestrial Networks: Recent Advances and Future Directionscited by 10
  7. 2025Fast and Cost-effective Speculative Edge-Cloud Decoding with Early Exitscited by 8
  8. 2025LightVLM: Acceleraing Large Multimodal Models with Pyramid Token Merging and KV Cache Compressioncited by 7
  9. 2025ActionFlow: A Pipelined Action Acceleration for Vision Language Models on Edgecited by 6
  10. 2025Kelle: Co-design KV Caching and eDRAM for Efficient LLM Serving in Edge Computingcited by 5
  11. 2025PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computingcited by 4
  12. 2025Are We There Yet? A Measurement Study of Efficiency for LLM Applications on Mobile Devicescited by 4
14 more
  1. 2025UserCentrix: An Agentic Memory-augmented AI Framework for Smart Spacescited by 4
  2. 2025Towards Pervasive Distributed Agentic Generative AI -- A State of The Artcited by 4
  3. 2025Accelerating Mobile Language Model via Speculative Decoding and NPU-Coordinated Executioncited by 4
  4. 2024Towards Edge General Intelligence via Large Language Models: Opportunities and Challengescited by 3
  5. 2025Edge General Intelligence Through World Models and Agentic AI: Fundamentals, Solutions, and Challengescited by 3
  6. 2025FastTTS: Accelerating Test-Time Scaling for Edge LLM Reasoningcited by 3
  7. 2025One-for-All Pruning: A Universal Model for Customized Compression of Large Language Modelscited by 2
  8. 2025Scaling behavior of large language models in emotional safety classification across sizes and taskscited by 1
  9. 2025PrivTune: Efficient and Privacy-Preserving Fine-Tuning of Large Language Models via Device-Cloud Collaborationcited by 1
  10. 2026WISP: Waste- and Interference-Suppressed Distributed Speculative LLM Serving at the Edge via Dynamic Drafting and SLO-Aware Batchingcited by 1
  11. 2026ConfigSpec: Profiling-Based Configuration Selection for Distributed Edge--Cloud Speculative LLM Servingcited by 1
  12. 2024Adaptive Pruning for Large Language Models with Structural Importance Awarenesscited by 0
  13. 2025SSFO: Self-Supervised Faithfulness Optimization for Retrieval-Augmented Generationcited by 0
  14. 2026AQLoRA: A Zero-Search Recipe for Fast Quantized LoRA Fine-Tuningcited by 0