# Are We There Yet? A Measurement Study of Efficiency for LLM Applications on Mobile Devices

arXiv: https://arxiv.org/abs/2504.00002 · HTML: https://mlsanity.com/paper/2504.00002/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 41 references that are in the map.
- First posted on arXiv 10 Mar 2025.

## Builds on (most cited first, 10 of 10)

- [AutoDroid: LLM-powered Task Automation in Android](https://mlsanity.com/paper/2308.15272/) (2023)
- [A Review on Edge Large Language Models: Design, Execution, and Applications](https://mlsanity.com/paper/2410.11845/) (2024)
- [Penetrative AI: Making LLMs Comprehend the Physical World](https://mlsanity.com/paper/2310.09605/) (2023)
- [Fast On-device LLM Inference with NPUs](https://mlsanity.com/paper/2407.05858/) (2024)
- [Explore, Select, Derive, and Recall: Augmenting LLM with Human-like Memory for Mobile Task Automation](https://mlsanity.com/paper/2312.03003/) (2023)
- [LLMSense: Harnessing LLMs for High-level Reasoning Over Spatiotemporal Sensor Traces](https://mlsanity.com/paper/2403.19857/) (2024)
- [EDGE-LLM: Enabling Efficient Large Language Model Adaptation on Edge Devices via Layerwise Unified Compression and Adaptive Layer Tuning and Voting](https://mlsanity.com/paper/2406.15758/) (2024)
- [Enabling On-Device Large Language Model Personalization with Self-Supervised Data Selection and Synthesis](https://mlsanity.com/paper/2311.12275/) (2023)
- [Boosting DNN Cold Inference on Edge Devices](https://mlsanity.com/paper/2206.07446/) (2022)
- [SensorBench: Benchmarking LLMs in Coding-Based Sensor Processing](https://mlsanity.com/paper/2410.10741/) (2024)

## Built on by (most cited first, 4 of 4)

- [Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Support](https://mlsanity.com/paper/2604.18302/) (2026)
- [Multi-Bitwidth Quantization for LLMs Using Additive Codebooks](https://mlsanity.com/paper/2606.12876/) (2026)
- [Can Edge-Deployable Vision-Language Models Identify Species?](https://mlsanity.com/paper/2609.11916/) (2026)
- [Evaluating Zero-Shot and One-Shot Adaptation of Small Language Models in Leader-Follower Interaction](https://mlsanity.com/paper/2602.23312/) (2026)

## Abstract (arXiv)

Recent advancements in large language models (LLMs) have prompted interest in deploying these models on mobile devices to enable new applications without relying on cloud connectivity. However, the efficiency constraints of deploying LLMs on resource-limited devices present significant challenges. In this paper, we conduct a comprehensive measurement study to evaluate the efficiency tradeoffs between mobile-based, edge-based, and cloud-based deployments for LLM applications. We implement AutoLife-Lite, a simplified LLM-based application that analyzes smartphone sensor data to infer user location and activity contexts. Our experiments reveal that: (1) Only small-size LLMs (<4B parameters) can run successfully on powerful mobile devices, though they exhibit quality limitations compared to larger models; (2) Model compression is effective in lower the hardware requirement, but may lead to significant performance degradation; (3) The latency to run LLMs on mobile devices with meaningful output is significant (>30 seconds), while cloud services demonstrate better time efficiency (<10 seconds); (4) Edge deployments offer intermediate tradeoffs between latency and model capabilities, with different results on CPU-based and GPU-based settings. These findings provide valuable insights for system designers on the current limitations and future directions for on-device LLM applications.