mlsanityData to 7 Oct 2026

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases

arXiv 2402.14905 ↗First posted 22 Feb 2024cs.LG cs.AI cs.CLcited by 216

Not on the map. Why?

Abstract (arXiv)

This paper addresses the growing need for efficient large language models (LLMs) on mobile devices, driven by increasing cloud costs and latency concerns. We focus on designing top-quality LLMs with fewer than a billion parameters, a practical choice for mobile deployment. Contrary to prevailing belief emphasizing the pivotal role of data and parameter quantity in determining model quality, our investigation underscores the significance of model architecture for sub-billion scale LLMs. Leveraging deep and thin architectures, coupled with embedding sharing and grouped-query attention mechanisms, we establish a strong baseline network denoted as MobileLLM, which attains a remarkable 2.7%/4.3% accuracy boost over preceding 125M/350M state-of-the-art models. Additionally, we propose an immediate block-wise weight-sharing approach with no increase in model size and only marginal latency overhead. The resultant models, denoted as MobileLLM-LS, demonstrate a further accuracy enhancement of 0.7%/0.8% than MobileLLM 125M/350M. Moreover, MobileLLM model family shows significant improvements compared to previous sub-billion models on chat benchmarks, and demonstrates close correctness to LLaMA-v2 7B in API calling tasks, highlighting the capability of small models for common on-device use cases.

Key facts. Cited by 216 papers in the map (data through 7 Oct 2026). Builds on 52 of its 65 references that are in the map. First posted on arXiv 22 Feb 2024.

Builds on

52 of 65 references.

References · 52

  1. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2020Scaling Laws for Neural Language Modelscited by 6,411
  7. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  11. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  12. 2023Qwen Technical Reportcited by 3,273
38 more
  1. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  2. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  3. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  4. 2018Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answeringcited by 2,114
  5. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  6. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,968
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  8. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  9. 2020Reformer: The Efficient Transformercited by 1,658
  10. 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  12. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  13. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  14. 2017RACE: Large-scale ReAding Comprehension Dataset From Examinationscited by 851
  15. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  16. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 776
  17. 2022Galactica: A Large Language Model for Sciencecited by 736
  18. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  19. 2016Language Modeling with Gated Convolutional Networkscited by 688
  20. 2024TinyLlama: An Open-Source Small Language Modelcited by 604
  21. 2023FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPUcited by 567
  22. 2023The Falcon Series of Open Language Modelscited by 526
  23. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  24. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 480
  25. 2020Efficient Content-Based Sparse Attention with Routing Transformerscited by 469
  26. 2021Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attentioncited by 468
  27. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  28. 2023Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruningcited by 408
  29. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  30. 2023SqueezeLLM: Dense-and-Sparse Quantizationcited by 285
  31. 2023LaMini-LM: A Diverse Herd of Distilled Models from Large-Scale Instructionscited by 124
  32. 2023Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Clustercited by 106
  33. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 65
  34. 2023QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Modelscited by 63
  35. 2023Baby Llama: knowledge distillation from an ensemble of teachers trained on a small dataset with no performance penaltycited by 57
  36. 2021NAS-BERT: Task-Agnostic and Adaptive-Size BERT Compression with Neural Architecture Searchcited by 55
  37. 2021Subformer: Exploring Weight Sharing for Parameter Efficiency in Generative Transformerscited by 42
  38. 2021Sliced Recursive Transformercited by 26

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 46; 2025: 93; 2026: 77.

216 papers.

Sort:

By area:

  1. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  2. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  3. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 527
  4. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  5. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approachcited by 372
  6. 2024LayerSkip: Enabling Early Exit Inference and Self-Speculative Decodingcited by 302
  7. 2025Reasoning with Latent Thoughts: On the Power of Looped Transformerscited by 223
  8. 2024A Survey on Efficient Inference for Large Language Modelscited by 197
  9. 2024Large Language Model-Brained GUI Agents: A Surveycited by 160
  10. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  11. 2024DeeR-VLA: Dynamic Inference of Multimodal Large Language Models for Efficient Robot Executioncited by 116
  12. 2024On-Device Language Models: A Comprehensive Reviewcited by 101
38 more
  1. 2024Relaxed Recursive Transformers: Effective Parameter Sharing with Layer-wise LoRAcited by 72
  2. 2025A Survey of Personalized Large Language Models: Progress and Future Directionscited by 65
  3. 2024APOLLO: SGD-like Memory, AdamW-level Performancecited by 50
  4. 2025KeyDiff: Key Similarity-Based KV Cache Eviction for Long-Context LLM Inference in Resource-Constrained Environmentscited by 49
  5. 2024Octopus v2: On-device language model for super agentcited by 45
  6. 2025ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantizationcited by 44
  7. 2025Distilling LLM Agent into Small Models with Retrieval and Code Toolscited by 41
  8. 2025Jet-Nemotron: Efficient Language Model with Post Neural Architecture Searchcited by 39
  9. 2024Q-GaLore: Quantized GaLore with INT4 Projection and Layer-Adaptive Low-Rank Gradientscited by 36
  10. 2024MELTing point: Mobile Evaluation of Language Transformerscited by 35
  11. 2025Seq vs Seq: An Open Suite of Paired Encoders and Decoderscited by 34
  12. 2024A Survey of Small Language Modelscited by 31
  13. 2025Parallel Scaling Law for Language Modelscited by 29
  14. 2025MiniRAG: Towards Extremely Simple Retrieval-Augmented Generationcited by 28
  15. 2024AgentMove: A Large Language Model based Agentic Framework for Zero-shot Next Location Predictioncited by 25
  16. 2025Encoder-Decoder Gemma: Improving the Quality-Efficiency Trade-Off via Adaptationcited by 25
  17. 2024Mipha: A Comprehensive Overhaul of Multimodal Assistant with Small Language Modelscited by 24
  18. 2025Agentic Plan Caching: Test-Time Memory for Fast and Cost-Efficient LLM Agentscited by 23
  19. 2025Encode, Think, Decode: Scaling test-time reasoning with recursive latent thoughtscited by 23
  20. 2025TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Modelscited by 19
  21. 2025The Rise of Small Language Models in Healthcare: A Comprehensive Surveycited by 18
  22. 2024HarmAug: Effective Data Augmentation for Knowledge Distillation of Safety Guard Modelscited by 17
  23. 2024Phantom of Latent for Large Language and Vision Modelscited by 15
  24. 2025A Token is Worth over 1,000 Tokens: Efficient Knowledge Distillation through Low-Rank Clonecited by 15
  25. 2025Turbo-VAED: Fast and Stable Transfer of Video-VAEs to Mobile Devicescited by 15
  26. 2025Nemotron-Flash: Towards Latency-Optimal Hybrid Small Language Modelscited by 15
  27. 2024MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Casescited by 13
  28. 2025CEReBrO: Compact Encoder for Representations of Brain Oscillations Using Efficient Alternating Attentioncited by 13
  29. 2024WDMoE: Wireless Distributed Mixture of Experts for Large Language Modelscited by 12
  30. 2025Recurrent Deep Differentiable Logic Gate Networkscited by 12
  31. 2024Deeploy: Enabling Energy-Efficient Deployment of Small Language Models On Heterogeneous Microcontrollerscited by 11
  32. 2025SensorChat: Answering Qualitative and Quantitative Questions during Long-Term Multimodal Sensor Interactionscited by 11
  33. 2025Slamming: Training a Speech Language Model on One GPU in a Daycited by 11
  34. 2024VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Modelscited by 10
  35. 2025EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Modelscited by 10
  36. 2026STEM: Scaling Transformers with Embedding Modulescited by 10
  37. 2024FoldGPT: Simple and Effective Large Language Model Compression Schemecited by 9
  38. 2025MobileRAG: A Fast, Memory-Efficient, and Energy-Efficient Method for On-Device RAGcited by 9

50 most cited of 216 All 216 (JSON)