# A Performance Evaluation of a Quantized Large Language Model on Various Smartphones

arXiv: https://arxiv.org/abs/2312.12472 · HTML: https://mlsanity.com/paper/2312.12472/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 12 references that are in the map.
- First posted on arXiv 19 Dec 2023.

## Builds on (most cited first, 9 of 9)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://mlsanity.com/paper/2301.00774/) (2023)
- [OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models](https://mlsanity.com/paper/2308.13137/) (2023)
- [Teaching Small Language Models to Reason](https://mlsanity.com/paper/2212.08410/) (2022)
- [A Survey on Model Compression for Large Language Models](https://mlsanity.com/paper/2308.07633/) (2023)
- [MiniLLM: On-Policy Distillation of Large Language Models](https://mlsanity.com/paper/2306.08543/) (2023)
- [In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Models](https://mlsanity.com/paper/2212.10670/) (2022)
- [8-bit Numerical Formats for Deep Neural Networks](https://mlsanity.com/paper/2206.02915/) (2022)
- [Serverless Computing: A Security Perspective](https://mlsanity.com/paper/2107.03832/) (2021)

## Built on by (most cited first, 7 of 7)

- [LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding](https://mlsanity.com/paper/2404.16710/) (2024)
- [On-Device Language Models: A Comprehensive Review](https://mlsanity.com/paper/2409.00088/) (2024)
- [Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latency](https://mlsanity.com/paper/2504.03360/) (2025)
- [Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deployment](https://mlsanity.com/paper/2501.03265/) (2025)
- [PalmBench: A Comprehensive Benchmark of Compressed Large Language Models on Mobile Platforms](https://mlsanity.com/paper/2410.05315/) (2024)
- [Medicine on the Edge: Comparative Performance Analysis of On-Device LLMs for Clinical Reasoning](https://mlsanity.com/paper/2502.08954/) (2025)
- [LLMs in Mobile Apps: Practices, Challenges, and Opportunities](https://mlsanity.com/paper/2502.15908/) (2025)

## Abstract (arXiv)

This paper explores the feasibility and performance of on-device large language model (LLM) inference on various Apple iPhone models. Amidst the rapid evolution of generative AI, on-device LLMs offer solutions to privacy, security, and connectivity challenges inherent in cloud-based models. Leveraging existing literature on running multi-billion parameter LLMs on resource-limited devices, our study examines the thermal effects and interaction speeds of a high-performing LLM across different smartphone generations. We present real-world performance results, providing insights into on-device inference capabilities.