mlsanityData to 7 Oct 2026

MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices

arXiv 2312.16886 ↗First posted 28 Dec 2023cs.CVcited by 120

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

We present MobileVLM, a competent multimodal vision language model (MMVLM) targeted to run on mobile devices. It is an amalgamation of a myriad of architectural designs and techniques that are mobile-oriented, which comprises a set of language models at the scale of 1.4B and 2.7B parameters, trained from scratch, a multimodal vision model that is pre-trained in the CLIP fashion, cross-modality interaction via an efficient projector. We evaluate MobileVLM on several typical VLM benchmarks. Our models demonstrate on par performance compared with a few much larger models. More importantly, we measure the inference speed on both a Qualcomm Snapdragon 888 CPU and an NVIDIA Jeston Orin GPU, and we obtain state-of-the-art performance of 21.5 tokens and 65.3 tokens per second, respectively. Our code will be made available at: https://github.com/Meituan-AutoML/MobileVLM.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 93 of its 125 references that are in the map. First posted on arXiv 28 Dec 2023.

Builds on

93 of 125 references.

References · 93

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  8. 2023Visual Instruction Tuningcited by 7,792
  9. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  10. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  11. 2020Scaling Laws for Neural Language Modelscited by 6,411
  12. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
38 more
  1. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  4. 2023Improved Baselines with Visual Instruction Tuningcited by 4,362
  5. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  8. 2021Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervisioncited by 3,848
  9. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  12. 2022Training Compute-Optimal Large Language Modelscited by 2,927
  13. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  14. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  15. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  16. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  17. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,646
  18. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  19. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  20. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  21. 2023MMBench: Is Your Multi-modal Model an All-around Player?cited by 2,105
  22. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  23. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  24. 2022Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answeringcited by 1,881
  25. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  26. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  27. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  28. 2020GLU Variants Improve Transformercited by 1,724
  29. 2019Towards VQA Models That Can Readcited by 1,696
  30. 2023Evaluating Object Hallucination in Large Vision-Language Modelscited by 1,696
  31. 2023MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Modelscited by 1,624
  32. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  33. 2019Root Mean Square Layer Normalizationcited by 1,425
  34. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,352
  35. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  36. 2021Conceptual 12M: Pushing Web-Scale Image-Text Pre-Training To Recognize Long-Tail Visual Conceptscited by 1,209
  37. 2023SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shotcited by 1,090
  38. 2021Perceiver: General Perception with Iterative Attentioncited by 1,089

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 37; 2025: 49; 2026: 34.

120 papers.

Sort:

By area:

  1. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  2. 2024Molmo and PixMo: Open Weights and Open Data for State-of-the-Art Vision-Language Modelscited by 321
  3. 2024TinyLLaVA: A Framework of Small-scale Large Multimodal Modelscited by 164
  4. 2024RoboMamba: Efficient Vision-Language-Action Model for Robotic Reasoning and Manipulationcited by 102
  5. 2024Cobra: Extending Mamba to Multi-Modal Large Language Model for Efficient Inferencecited by 97
  6. 2024Efficient Multimodal Large Language Models: A Surveycited by 95
  7. 2024VL-Mamba: Exploring State Space Models for Multimodal Learningcited by 93
  8. 2024Vision-and-Language Navigation Today and Tomorrow: A Survey in the Era of Foundation Modelscited by 79
  9. 2024Small Language Model Meets with Reinforced Vision Vocabularycited by 43
  10. 2024LLaVA-MoD: Making LLaVA Tiny via MoE Knowledge Distillationcited by 43
  11. 2025Manipulating Multimodal Agents via Cross-Modal Prompt Injectioncited by 42
  12. 2024From Redundancy to Relevance: Information Flow in LVLMs Across Reasoning Taskscited by 37
38 more
  1. 2025A Survey of Multimodal Retrieval-Augmented Generationcited by 37
  2. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Modelscited by 35
  3. 2025AutoDrive-R2: Incentivizing Reasoning and Self-Reflection Capacity for VLA Model in Autonomous Drivingcited by 32
  4. 2024How Far Are We From AGI: Are LLMs All We Need?cited by 31
  5. 2024African or European Swallow? Benchmarking Large Vision-Language Models for Fine-Grained Object Classificationcited by 31
  6. 2024InstructSeg: Unifying Instructed Visual Segmentation with Multi-modal Large Language Modelscited by 31
  7. 2024WiseAD: Knowledge Augmented End-to-End Autonomous Driving with Vision-Language Modelcited by 28
  8. 2024[CLS] Token Tells Everything Needed for Training-free Efficient MLLMscited by 26
  9. 2024BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devicescited by 25
  10. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  11. 2024Align-KD: Distilling Cross-Modal Alignment Knowledge for Mobile Vision-Language Model Enhancementcited by 21
  12. 2025Multi-Layer Visual Feature Fusion in Multimodal LLMs: Methods, Analysis, and Best Practicescited by 21
  13. 2025InternVL-X: Advancing and Accelerating InternVL Series with Efficient Visual Token Compressioncited by 21
  14. 2024Memory-Space Visual Prompting for Efficient Vision-Language Fine-Tuningcited by 20
  15. 2024ST3: Accelerating Multimodal Large Language Model by Spatial-Temporal Visual Token Trimmingcited by 20
  16. 2025A Survey on Efficient Vision-Language Modelscited by 19
  17. 2024SEA: Supervised Embedding Alignment for Token-Level Visual-Textual Integration in MLLMscited by 17
  18. 2024Progressive Mixed-Precision Decoding for Efficient LLM Inferencecited by 17
  19. 2024Hardware-Aware Parallel Prompt Decoding for Memory-Efficient Acceleration of LLM Inferencecited by 16
  20. 2025Multimodal Large Language Models for Medicine: A Comprehensive Surveycited by 16
  21. 2026Vision-aligned Latent Reasoning for Multi-modal Large Language Modelcited by 16
  22. 2025Towards Enhanced Image Generation Via Multi-modal Chain of Thought in Unified Generative Modelscited by 14
  23. 2024MobileAIBench: Benchmarking LLMs and LMMs for On-Device Use Casescited by 13
  24. 2024Rethinking Visual Prompting for Multimodal Large Language Models with External Knowledgecited by 13
  25. 2025Device-Cloud Collaborative LLM Inference with Multi-Modal, Multi-Task, Multi-Turn Conversationscited by 13
  26. 2025Active Perception Agent for Omnimodal Audio-Video Understandingcited by 13
  27. 2025GhostEI-Bench: Do Mobile Agents Resilience to Environmental Injection in Dynamic On-Device Environments?cited by 12
  28. 2024Deeploy: Enabling Energy-Efficient Deployment of Small Language Models On Heterogeneous Microcontrollerscited by 11
  29. 2025Eve: Efficient Multimodal Vision Language Models with Elastic Visual Expertscited by 11
  30. 2024VisualRWKV: Exploring Recurrent Neural Networks for Visual Language Modelscited by 10
  31. 2024ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2cited by 10
  32. 2024FoPru: Focal Pruning for Efficient Large Vision-Language Modelscited by 10
  33. 2024VLsI: Verbalized Layers-to-Interactions from Large to Small Vision Language Modelscited by 10
  34. 2024FlashSloth: Lightning Multimodal Large Language Models via Embedded Visual Compressioncited by 10
  35. 2024LLaVA-Ultra: Large Chinese Language and Vision Assistant for Ultrasoundcited by 8
  36. 2025TransBench: Breaking Barriers for Transferable Graphical User Interface Agents in Dynamic Digital Environmentscited by 8
  37. 2026Shallow-π: Knowledge Distillation for Flow-based VLAscited by 7
  38. 2025PLPHP: Per-Layer Per-Head Vision Token Pruning for Efficient Large Vision-Language Modelscited by 6

50 most cited of 120 All 120 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024MobileVLM V2: Faster and Stronger Baseline for Vision Language Model
  2. 2025SmolVLM: Redefining small and efficient multimodal models
  3. 2026Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs
  4. 2024Mini-InternVL: A Flexible-Transfer Pocket Multimodal Model with 5% Parameters and 90% Performance
  5. 2025MagicVL-2B: Empowering Vision-Language Models on Mobile Devices with Lightweight Visual Encoders via Curriculum Learning
  6. 2024Xmodel-VLM: A Simple Baseline for Multimodal Vision Language Model
  7. 2024Training a Vision Language Model as Smartphone Assistant
  8. 2025LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environments