# A Review on Edge Large Language Models: Design, Execution, and Applications

arXiv: https://arxiv.org/abs/2410.11845 · HTML: https://mlsanity.com/paper/2410.11845/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Sep 2024.

## Built on by (most cited first, 30 of 100)

- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [Balanced Token Pruning: Accelerating Vision Language Models Beyond Local Optimization](https://mlsanity.com/paper/2505.22038/) (2025)
- [Xiaomi MiMo-VL-Miloco Technical Report](https://mlsanity.com/paper/2512.17436/) (2025)
- [A Survey on Collaborative Mechanisms Between Large and Small Language Models](https://mlsanity.com/paper/2505.07460/) (2025)
- [Object Detection with Multimodal Large Vision-Language Models: An In-depth Review](https://mlsanity.com/paper/2508.19294/) (2025)
- [LLM-Oriented Token-Adaptive Knowledge Distillation](https://mlsanity.com/paper/2510.11615/) (2025)
- [LittleBit: Ultra Low-Bit Quantization via Latent Factorization](https://mlsanity.com/paper/2506.13771/) (2025)
- [A Strategic Coordination Framework of Small LLMs Matches Large LLMs in Data Synthesis](https://mlsanity.com/paper/2504.12322/) (2025)
- [Small Models, Big Tasks: An Exploratory Empirical Study on Small Language Models for Function Calling](https://mlsanity.com/paper/2504.19277/) (2025)
- [SlimCaching: Edge Caching of Mixture-of-Experts for Distributed Inference](https://mlsanity.com/paper/2507.06567/) (2025)
- [AVA: Towards Agentic Video Analytics with Vision Language Models](https://mlsanity.com/paper/2505.00254/) (2025)
- [Towards Efficient Multi-LLM Inference: Characterization and Analysis of LLM Routing and Hierarchical Techniques](https://mlsanity.com/paper/2506.06579/) (2025)
- [RooflineBench: A Benchmarking Framework for On-Device LLMs via Roofline Analysis](https://mlsanity.com/paper/2602.11506/) (2026)
- [PLM: Efficient Peripheral Language Models Hardware-Co-Designed for Ubiquitous Computing](https://mlsanity.com/paper/2503.12167/) (2025)
- [Are We There Yet? A Measurement Study of Efficiency for LLM Applications on Mobile Devices](https://mlsanity.com/paper/2504.00002/) (2025)
- [Token Level Routing Inference System for Edge Devices](https://mlsanity.com/paper/2504.07878/) (2025)
- [RedAHD: Reduction-Based End-to-End Automatic Heuristic Design with Large Language Models](https://mlsanity.com/paper/2505.20242/) (2025)
- [DP-FedLoRA: Privacy-Enhanced Federated Fine-Tuning for On-Device Large Language Models](https://mlsanity.com/paper/2509.09097/) (2025)
- [ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Models](https://mlsanity.com/paper/2510.27256/) (2025)
- [T-SAR: A Full-Stack Co-design for CPU-Only Ternary LLM Inference via In-Place SIMD ALU Reorganization](https://mlsanity.com/paper/2511.13676/) (2025)
- [ECG-Agent: On-Device Tool-Calling Agent for ECG Multi-Turn Dialogue](https://mlsanity.com/paper/2601.20323/) (2026)
- [Hierarchical Memory Orchestration for Personalized Persistent Agents](https://mlsanity.com/paper/2604.01670/) (2026)
- [FastTTS: Accelerating Test-Time Scaling for Edge LLM Reasoning](https://mlsanity.com/paper/2509.00195/) (2025)
- [Federate the Router: Learning Language Model Routers with Sparse and Decentralized Evaluations](https://mlsanity.com/paper/2601.22318/) (2026)
- [Multi-Agent Home Energy Management Assistant](https://mlsanity.com/paper/2602.15219/) (2026)
- [A Pipelined Collaborative Speculative Decoding Framework for Efficient Edge-Cloud LLM Inference](https://mlsanity.com/paper/2603.19133/) (2026)
- [Skill is Not One-Size-Fits-All: Model-Aware Skill Alignment for LLM Agents](https://mlsanity.com/paper/2605.30723/) (2026)
- [OrderMoE: An expert similarity driven distributed edge MoE inference](https://mlsanity.com/paper/2607.17154/) (2026)
- [Distributed Retrieval-Augmented Generation](https://mlsanity.com/paper/2505.00443/) (2025)
- [P/D-Device: Disaggregated Large Language Model between Cloud and Devices](https://mlsanity.com/paper/2508.09035/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have revolutionized natural language processing with their exceptional understanding, synthesizing, and reasoning capabilities. However, deploying LLMs on resource-constrained edge devices presents significant challenges due to computational limitations, memory constraints, and edge hardware heterogeneity. This survey provides a comprehensive overview of recent advancements in edge LLMs, covering the entire lifecycle: from resource-efficient model design and pre-deployment strategies to runtime inference optimizations. It also explores on-device applications across various domains. By synthesizing state-of-the-art techniques and identifying future research directions, this survey bridges the gap between the immense potential of LLMs and the constraints of edge computing.