mlsanityData to 14 Oct 2026

Mobile Foundation Model as Firmware

arXiv 2308.14363 ↗First posted 28 Aug 2023cs.AIcited by 25

Not on the map. Why?

Abstract (arXiv)

In today's landscape, smartphones have evolved into hubs for hosting a multitude of deep learning models aimed at local execution. A key realization driving this work is the notable fragmentation among these models, characterized by varied architectures, operators, and implementations. This fragmentation imposes a significant burden on the comprehensive optimization of hardware, system settings, and algorithms. Buoyed by the recent strides in large foundation models, this work introduces a pioneering paradigm for mobile AI: a collaborative management approach between the mobile OS and hardware, overseeing a foundational model capable of serving a broad spectrum of mobile AI tasks, if not all. This foundational model resides within the NPU and remains impervious to app or OS revisions, akin to firmware. Concurrently, each app contributes a concise, offline fine-tuned "adapter" tailored to distinct downstream tasks. From this concept emerges a concrete instantiation known as sys. It amalgamates a curated selection of publicly available Large Language Models (LLMs) and facilitates dynamic data flow. This concept's viability is substantiated through the creation of an exhaustive benchmark encompassing 38 mobile AI tasks spanning 50 datasets, including domains such as Computer Vision (CV), Natural Language Processing (NLP), audio, sensing, and multimodal inputs. Spanning this benchmark, sys unveils its impressive performance. It attains accuracy parity in 85% of tasks, demonstrates improved scalability in terms of storage and memory, and offers satisfactory inference speed on Commercial Off-The-Shelf (COTS) mobile devices fortified with NPU support. This stands in stark contrast to task-specific models tailored for individual applications.

Key facts. Cited by 25 papers in the map (data through 14 Oct 2026). Builds on 61 of its 124 references that are in the map. First posted on arXiv 28 Aug 2023.

Builds on

61 of 124 references.

References · 61

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  5. 2020Language Models are Few-Shot Learnerscited by 32,419
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  7. 2023GPT-4 Technical Reportcited by 18,396
  8. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  9. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  10. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  11. 2023Segment Anythingcited by 8,234
  12. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
38 more
  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  2. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  3. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
  4. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  5. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  6. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  8. 2021BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-modelscited by 1,245
  9. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  10. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  11. 2023ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generationcited by 1,045
  12. 2019How Can We Know What Language Models Know?cited by 959
  13. 2023A Survey on Multimodal Large Language Modelscited by 909
  14. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 843
  15. 2021AST: Audio Spectrogram Transformercited by 643
  16. 2022Detecting Twenty-thousand Classes using Image-level Supervisioncited by 614
  17. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  18. 2015Bidirectional LSTM-CRF Models for Sequence Taggingcited by 546
  19. 2023Language Is Not All You Need: Aligning Perception with Language Modelscited by 543
  20. 2020MobileBERT: a Compact Task-Agnostic BERT for Resource-Limited Devicescited by 482
  21. 2021Time-Series Representation Learning via Temporal and Contextual Contrastingcited by 345
  22. 2023PandaGPT: One Model To Instruction-Follow Them Allcited by 334
  23. 2022Images Speak in Images: A Generalist Painter for In-Context Visual Learningcited by 318
  24. 2020BERT Loses Patience: Fast and Robust Inference with Early Exitcited by 273
  25. 2021Global Filter Networks for Image Classificationcited by 260
  26. 2021Mobile-Former: Bridging MobileNet and Transformercited by 235
  27. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 234
  28. 2023Any-to-Any Generation via Composable Diffusioncited by 204
  29. 2017MUTAN: Multimodal Tucker Fusion for Visual Question Answeringcited by 137
  30. 2021QuantumNAS: Noise-Adaptive Search for Robust Quantum Circuitscited by 98
  31. 2021Adaptive Inference through Early-Exit Networks: Design, Challenges and Directionscited by 75
  32. 2021MobRecon: Mobile-Friendly Hand Mesh Reconstruction from Monocular Imagecited by 74
  33. 2018Show, Control and Tell: A Framework for Generating Controllable and Grounded Captionscited by 65
  34. 2022ViTPose++: Vision Transformer for Generic Body Pose Estimationcited by 65
  35. 2017DeepCache: Principled Cache for Mobile Deep Visioncited by 57
  36. 2021Semantic Image Mattingcited by 55
  37. 2018A First Look at Deep Learning Apps on Smartphonescited by 47
  38. 2020Improving on-device speaker verification using federated learning with privacycited by 30

50 most cited of 61 All 61 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 8; 2026: 3.

25 papers.

Sort:

By area:

  1. 2024Small Language Models: Survey, Measurements, and Insightscited by 112
  2. 2024A Survey of Resource-efficient LLM and Multimodal Foundation Modelscited by 86
  3. 2024Fast On-device LLM Inference with NPUscited by 62
  4. 2025Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Modelscited by 46
  5. 2024AutoDroid-V2: Boosting SLM-based GUI Agents via Code Generationcited by 34
  6. 2024Computational Experiments Meet Large Language Model Based Agents: A Survey and Perspectivecited by 11
  7. 2024TinyLLM: A Framework for Training and Deploying Language Models at the Edge Computerscited by 9
  8. 2024Babel: A Scalable Pre-trained Model for Multi-Modal Sensing via Expandable Modality Alignmentcited by 8
  9. 2025AugFL: Augmenting Federated Learning with Pretrained Modelscited by 6
  10. 2024DroidCall: A Dataset for LLM-powered Android Intent Invocationcited by 5
  11. 2025Foundation Models for CPS-IoT: Opportunities and Challengescited by 5
  12. 2025Semantic Communications with Computer Vision Sensing for Edge Video Transmissioncited by 5
13 more
  1. 2024UoMo: A Universal Model of Mobile Traffic Forecasting for Wireless Network Optimizationcited by 4
  2. 2024A Scene-aware Models Adaptation Scheme for Cross-scene Online Inference on Mobile Devicescited by 3
  3. 2024Elastic On-Device LLM Servicecited by 3
  4. 2024Recall: Empowering Multimodal Embedding for Edge Devicescited by 3
  5. 2024Towards Edge General Intelligence via Large Language Models: Opportunities and Challengescited by 3
  6. 2025RAP: Runtime Adaptive Pruning for LLM Inferencecited by 3
  7. 2026FedMomentum: Preserving LoRA Training Momentum in Federated Fine-Tuningcited by 3
  8. 2025P/D-Device: Disaggregated Large Language Model between Cloud and Devicescited by 2
  9. 2025Vec-LUT: Vector Table Lookup for Parallel Ultra-Low-Bit LLM Inference on Edge Devicescited by 2
  10. 2026Parameter-Free Encoders Remain Viable for RDB Foundation Modelscited by 2
  11. 2024Resource-Efficient Personal Large Language Models Fine-Tuning with Collaborative Edge Computingcited by 1
  12. 2025Optimizing Storage Overhead of User Behavior Log for ML-embedded Mobile Appscited by 1
  13. 2026Hydra: Phase-Aware Workload Characterization of LLM Inference across Edge SoC Generations, Backends, and Quantization Levelscited by 1