mlsanityData to 14 Oct 2026

Are We There Yet? A Measurement Study of Efficiency for LLM Applications on Mobile Devices

arXiv 2504.00002 ↗First posted 10 Mar 2025cs.PF cs.AI cs.HCcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent advancements in large language models (LLMs) have prompted interest in deploying these models on mobile devices to enable new applications without relying on cloud connectivity. However, the efficiency constraints of deploying LLMs on resource-limited devices present significant challenges. In this paper, we conduct a comprehensive measurement study to evaluate the efficiency tradeoffs between mobile-based, edge-based, and cloud-based deployments for LLM applications. We implement AutoLife-Lite, a simplified LLM-based application that analyzes smartphone sensor data to infer user location and activity contexts. Our experiments reveal that: (1) Only small-size LLMs (<4B parameters) can run successfully on powerful mobile devices, though they exhibit quality limitations compared to larger models; (2) Model compression is effective in lower the hardware requirement, but may lead to significant performance degradation; (3) The latency to run LLMs on mobile devices with meaningful output is significant (>30 seconds), while cloud services demonstrate better time efficiency (<10 seconds); (4) Edge deployments offer intermediate tradeoffs between latency and model capabilities, with different results on CPU-based and GPU-based settings. These findings provide valuable insights for system designers on the current limitations and future directions for on-device LLM applications.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 10 of its 41 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

10 of 41 references.

References · 10

  1. 2023AutoDroid: LLM-powered Task Automation in Androidcited by 184
  2. 2024A Review on Edge Large Language Models: Design, Execution, and Applicationscited by 100
  3. 2023Penetrative AI: Making LLMs Comprehend the Physical Worldcited by 64
  4. 2024Fast On-device LLM Inference with NPUscited by 62
  5. 2023Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automationcited by 58
  6. 2024LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Tracescited by 28
  7. 2024EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Votingcited by 27
  8. 2023Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesiscited by 20
  9. 2022Boosting DNN Cold Inference on Edge Devicescited by 6
  10. 2024SensorBench: Benchmarking LLMs in Coding-Based Sensor Processingcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Supportcited by 2
  2. 2026Multi-Bitwidth Quantization for LLMs Using Additive Codebookscited by 1
  3. 2026Can Edge-Deployable Vision-Language Models Identify Species?cited by 1
  4. 2026Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interactioncited by 0