mlsanityData to 14 Oct 2026

EdgeProfiler: A Fast Profiling Framework for Lightweight LLMs on Edge Using Analytical Model

arXiv 2506.09061 ↗First posted 6 Jun 2025cs.DC cs.AI cs.PFcited by 2

Not on the map. Why?

Abstract (arXiv)

This paper introduces EdgeProfiler, a fast profiling framework designed for evaluating lightweight Large Language Models (LLMs) on edge systems. While LLMs offer remarkable capabilities in natural language understanding and generation, their high computational, memory, and power requirements often confine them to cloud environments. EdgeProfiler addresses these challenges by providing a systematic methodology for assessing LLM performance in resource-constrained edge settings. The framework profiles compact LLMs, including TinyLLaMA, Gemma3.1B, Llama3.2-1B, and DeepSeek-r1-1.5B, using aggressive quantization techniques and strict memory constraints. Analytical modeling is used to estimate latency, FLOPs, and energy consumption. The profiling reveals that 4-bit quantization reduces model memory usage by approximately 60-70%, while maintaining accuracy within 2-5% of full-precision baselines. Inference speeds are observed to improve by 2-3x compared to FP16 baselines across various edge devices. Power modeling estimates a 35-50% reduction in energy consumption for INT4 configurations, enabling practical deployment on hardware such as Raspberry Pi 4/5 and Jetson Orin Nano Super. Our findings emphasize the importance of efficient profiling tailored to lightweight LLMs in edge environments, balancing accuracy, energy efficiency, and computational feasibility.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 18 of its 22 references that are in the map. First posted on arXiv 6 Jun 2025.

Builds on

18 of 22 references.

References · 18

  1. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  2. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  3. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  6. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 906
  7. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 815
  8. 2019Learned Step Size Quantizationcited by 628
  9. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 526
  10. 2021A White Paper on Neural Network Quantizationcited by 472
  11. 2024BioMistral: A Collection of Open-Source Pretrained Large Language Models for Medical Domainscited by 301
  12. 2024MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Casescited by 223
6 more
  1. 2023LLM in a flash: Efficient Large Language Model Inference with Limited Memorycited by 177
  2. 2023Efficient LLM Inference on CPUscited by 24
  3. 2024LLM-Assisted Crisis Management: Building Advanced LLM Platforms for Effective Emergency Response and Public Collaborationcited by 21
  4. 2024Empirical Guidelines for Deploying LLMs onto Resource-constrained Edge Devicescited by 17
  5. 2024Deeploy: Enabling Energy-Efficient Deployment of Small Language Models On Heterogeneous Microcontrollerscited by 11
  6. 2024On-Device LLMs for SMEs: Challenges and Opportunitiescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025Dissecting Transformers: A CLEAR Perspective towards Green AIcited by 1
  2. 2026Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levelscited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency
  2. 2025MNN-LLM: A Generic Inference Engine for Fast Large Language Model Deployment on Mobile Devices
  3. 2026Profiling Lightweight Large Language Models
  4. 2025A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources
  5. 2024EdgeShard: Efficient LLM Inference via Collaborative Edge Computing
  6. 2025Sometimes Painful but Certainly Promising: Feasibility and Trade-offs of Language Model Inference at the Edge
  7. 2025EdgeLoRA: An Efficient Multi-Tenant LLM Serving System on Edge Devices
  8. 2024EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Voting