mlsanityData to 7 Oct 2026

A Performance Evaluation of a Quantized Large Language Model on Various Smartphones

arXiv 2312.12472 ↗First posted 19 Dec 2023cs.LG cs.AI cs.PFcited by 7

Not on the map. Why?

Abstract (arXiv)

This paper explores the feasibility and performance of on-device large language model (LLM) inference on various Apple iPhone models. Amidst the rapid evolution of generative AI, on-device LLMs offer solutions to privacy, security, and connectivity challenges inherent in cloud-based models. Leveraging existing literature on running multi-billion parameter LLMs on resource-limited devices, our study examines the thermal effects and interaction speeds of a high-performing LLM across different smartphone generations. We present real-world performance results, providing insights into on-device inference capabilities.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 9 of its 12 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

9 of 12 references.

References · 9

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  3. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  4. 2022Teaching Small Language Models to Reasoncited by 279
  5. 2023A Survey on Model Compression for Large Language Modelscited by 263
  6. 2023MiniLLM: On-Policy Distillation of Large Language Modelscited by 129
  7. 2022In-context Learning Distillation: Transferring Few-shot Learning Ability of Pre-trained Language Modelscited by 36
  8. 20228-bit Numerical Formats for Deep Neural Networkscited by 23
  9. 2021Serverless Computing: A Security Perspectivecited by 3

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 4.

7 papers.

  1. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  2. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
  3. 2025Sustainable LLM Inference for Edge AI: Evaluating Quantized LLMs for Energy Efficiency, Output Accuracy, and Inference Latencycited by 32
  4. 2025Cognitive Edge Computing: A Comprehensive Survey on Optimizing Large Models and AI Agents for Pervasive Deploymentcited by 10
  5. 2024PalmBench: A Comprehensive Benchmark of Compressed Large Language Models on Mobile Platformscited by 7
  6. 2025Medicine on the Edge: Comparative Performance Analysis of On-Device LLMs for Clinical Reasoningcited by 3
  7. 2025LLMs in Mobile Apps: Practices, Challenges, and Opportunitiescited by 0