mlsanityData to 14 Oct 2026

Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models

arXiv 2503.06027 ↗First posted 8 Mar 2025cs.AI cs.LG cs.NIcited by 46

Not on the map. Why?

Abstract (arXiv)

The rapid advancement of artificial intelligence (AI) technologies has led to an increasing deployment of AI models on edge and terminal devices, driven by the proliferation of the Internet of Things (IoT) and the need for real-time data processing. This survey comprehensively explores the current state, technical challenges, and future trends of on-device AI models. We define on-device AI models as those designed to perform local data processing and inference, emphasizing their characteristics such as real-time performance, resource constraints, and enhanced data privacy. The survey is structured around key themes, including the fundamental concepts of AI models, application scenarios across various domains, and the technical challenges faced in edge environments. We also discuss optimization and implementation strategies, such as data preprocessing, model compression, and hardware acceleration, which are essential for effective deployment. Furthermore, we examine the impact of emerging technologies, including edge computing and foundation models, on the evolution of on-device AI models. By providing a structured overview of the challenges, solutions, and future directions, this survey aims to facilitate further research and application of on-device AI, ultimately contributing to the advancement of intelligent systems in everyday life.

Key facts. Cited by 46 papers in the map (data through 14 Oct 2026). Builds on 99 of its 286 references that are in the map. First posted on arXiv 8 Mar 2025.

Builds on

99 of 286 references.

References · 99

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  7. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  8. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  9. 2019Searching for MobileNetV3cited by 1,994
  10. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  11. 2018DARTS: Differentiable Architecture Searchcited by 1,798
  12. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,200
38 more
  1. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,136
  2. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,135
  3. 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,101
  4. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  5. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 787
  6. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 673
  7. 2021MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformercited by 642
  8. 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 497
  9. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
  10. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 443
  11. 2020HRank: Filter Pruning using High-Rank Feature Mapcited by 292
  12. 2020DynaBERT: Dynamic BERT with Adaptive Width and Depthcited by 251
  13. 2019Edge Intelligence: Paving the Last Mile of Artificial Intelligence with Edge Computingcited by 228
  14. 2023Data-centric Artificial Intelligence: A Surveycited by 203
  15. 2022A Fast Post-Training Pruning Framework for Transformerscited by 162
  16. 2019Edge AI: On-Demand Accelerating Deep Neural Network Inference via Edge Computingcited by 152
  17. 2021Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspectivecited by 116
  18. 2024On-Device Language Models: A Comprehensive Reviewcited by 104
  19. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 100
  20. 2017Soft Weight-Sharing for Neural Network Compressioncited by 99
  21. 2021Edge Artificial Intelligence for 6G: Vision, Enabling Technologies, and Applicationscited by 97
  22. 2020Communication-Efficient Edge AI: Algorithms and Systemscited by 96
  23. 2019BottleNet++: An End-to-End Approach for Feature Compression in Device-Edge Co-Inference Systemscited by 95
  24. 2018Wireless Network Intelligence at the Edgecited by 92
  25. 2019Machine Learning at the Network Edge: A Surveycited by 84
  26. 2018Privacy Preserving Machine Learning: Threats and Solutionscited by 79
  27. 2019Edge Intelligence: The Confluence of Edge Computing and Artificial Intelligencecited by 76
  28. 2020PatDNN: Achieving Real-Time DNN Execution on Mobile Devices with Pattern-based Weight Pruningcited by 76
  29. 2020EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inferencecited by 70
  30. 2019On-Device Machine Learning: An Algorithms and Learning Theory Perspectivecited by 63
  31. 2021Privacy-Preserving Machine Learning: Methods, Challenges and Directionscited by 58
  32. 2020Learning Low-rank Deep Neural Networks via Singular Vector Orthogonality Regularization and Singular Value Sparsificationcited by 53
  33. 2021Active label cleaning for improved dataset quality under resource constraintscited by 44
  34. 2015Structured Transforms for Small-Footprint Deep Learningcited by 42
  35. 2018Deep k-Means: Re-Training and Parameter Sharing with Harder Cluster Assignments for Compressing Deep Convolutionscited by 33
  36. 2020FSpiNN: An Optimization Framework for Memory- and Energy-Efficient Spiking Neural Networkscited by 32
  37. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applicationscited by 32
  38. 2022DropNet: Reducing Neural Network Complexity via Iterative Pruningcited by 29

50 most cited of 99 All 99 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 30.

46 papers.

Sort:

By area:

  1. 2025From Tiny Machine Learning to Tiny Deep Learning: A Surveycited by 5
  2. 2025Automating Versatile Time-Series Analysis with Tiny Transformers on Embedded FPGAscited by 4
  3. 2025ECVL-ROUTER: Scenario-Aware Routing for Vision-Language Modelscited by 4
  4. 2026Performance Analysis of Edge and In-Sensor AI Processors: A Comparative Reviewcited by 4
  5. 2025AI for Regulatory Affairs: Balancing Accuracy, Interpretability, and Computational Cost in Medical Device Classificationcited by 3
  6. 2025Embodied AI with Foundation Models for Mobile Service Robots: A Systematic Reviewcited by 3
  7. 2026Future of Edge AI in biodiversity monitoringcited by 3
  8. 2026Unsupervised Confidence Calibration for Reasoning LLMs from a Single Generationcited by 3
  9. 2026ReMoE: Boosting Expert Reuse through Router Fine-Tuning in Memory-Constrained MoE LLM Inferencecited by 3
  10. 2025EdgeWisePersona: A Dataset for On-Device User Profiling from Natural Language Interactionscited by 2
  11. 2025Edge Artificial Intelligence: A Systematic Review of Evolution, Taxonomic Frameworks, and Future Horizonscited by 2
  12. 2026MosaicThinker: On-Device Visual Spatial Reasoning for Embodied AI via Iterative Construction of Space Representationcited by 2
34 more
  1. 2025Mobile-MMLU: A Mobile Intelligence Language Understanding Benchmarkcited by 1
  2. 2025Comparative Evaluation of ChatGPT and DeepSeek Across Key NLP Tasks: Strengths, Weaknesses, and Domain-Specific Performancecited by 1
  3. 2026ECR: Manifold-Guided Semantic Cues for Compact Language Modelscited by 1
  4. 2026GLIMPSE : Real-Time Text Recognition and Contextual Understanding for VQA in Wearablescited by 1
  5. 2026The Internet of Physical AI Agents: Interoperability, Longevity, and the Cost of Getting It Wrongcited by 1
  6. 2026WISV: Wireless-Informed Semantic Verification for Distributed Speculative Decoding in Device-Edge LLM Inferencecited by 1
  7. 2026When NPUs Are Not Always Faster: A Stage-Level Analysis of Mobile LLM Inferencecited by 1
  8. 2025Distilling Empathy from Large Language Modelscited by 0
  9. 2025Hot-Swap MarkBoard: An Efficient Black-box Watermarking Approach for Large-scale Model Distributioncited by 0
  10. 2025CSRAP: Enhanced Canvas Attention Scheduling for Real-Time Mission Critical Perceptioncited by 0
  11. 2025Beyond ImageNet: Understanding Cross-Dataset Robustness of Lightweight Vision Modelscited by 0
  12. 2025FiCABU: A Fisher-Based, Context-Adaptive Machine Unlearning Processor for Edge AIcited by 0
  13. 2025AutoTailor: Automatic and Efficient Adaptive Model Deployment for Diverse Edge Devicescited by 0
  14. 2025Dora: QoE-Aware Hybrid Parallelism for Distributed Edge AIcited by 0
  15. 2026MD-AirComp+: Adaptive Quantization for Blind Massive Digital Over-the-Air Computationcited by 0
  16. 2026Storing Less, Finding More: How Novelty Filtering Improves Cross-Modal Retrieval on Edge Camerascited by 0
  17. 2026MLFCIL: A Multi-Level Forgetting Mitigation Framework for Federated Class-Incremental Learning in LEO Satellitescited by 0
  18. 2026On Emotion-Sensitive Decision Making of Small Language Model Agentscited by 0
  19. 2026A Comparative Study of CNN Optimization Methods for Edge AI: Exploring the Role of Early Exitscited by 0
  20. 2026Real-Time Cellist Postural Evaluation With On-Device Computer Visioncited by 0
  21. 2026FFR: Forward-Forward Learning for Regressioncited by 0
  22. 2026TileFuse: A Fused Mixed-Precision Kernel Library for Efficient Quantized LLM Inference on AMD NPUscited by 0
  23. 2026CONCORD: Asynchronous Sparse Aggregation for Device-Cloud RAG under Document Isolationcited by 0
  24. 2026Sustainable Face Recognition on Low-Power Devices with VQ-VAE Embeddingscited by 0
  25. 2026ConCise: Training-Free Conclusion-Chain State Compression for Cost-Efficient Multi-Step RAG Servicescited by 0
  26. 2026Harvesting AI Computation at the Edge via Generic Approximationcited by 0
  27. 2026Little Brains, Big Feats: Exploring Compact Language Modelscited by 0
  28. 2026NanoVSR: Towards Real-Time Video Super-Resolution on Edge Devicescited by 0
  29. 2026ExaGEMM: Exploration Framework for CPU-Driven ML Inference via Associative In-Register Computing for Low-Bit GEMMcited by 0
  30. 2026Position: AI Lock-In Is in Progress, and We Must Be Preparedcited by 0
  31. 2026FedADB: Class Anchor-Driven Dual-Branch Federated Learning for Mitigating Forgettingcited by 0
  32. 2026Large Models for Small Devices: Recent Advances and Empirical Analysis of Edge AI Deploymentcited by 0
  33. 2026FedLTLib: A Comprehensive Benchmark for Federated Long-Tail Learningcited by 0
  34. 2026BaCP: Backbone Contrastive Pruning for Preserving Representations in Extremely Sparse Neural Networkscited by 0