mlsanityData to 14 Oct 2026

Llama-Mobile: Efficient 2.7-Bit Quantization of VLMs

arXiv 2608.21134 ↗First posted 21 Aug 2026cs.CV cs.LGcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

Deploying vision-language models (VLMs) on mobile devices is challenging due to their significant memory and compute requirements. We present a framework for quantizing VLMs for efficient inference on resource-constrained hardware. Our approach combines a quantization pipeline that uses the model itself to generate training data and does not require access to the training setup, with a novel 2.7-bit-per-parameter format supporting efficient execution on Arm CPUs. We validate our approach by compressing the Llama 3.2 11B Vision Instruct model to 3.7 GB with 8-bit activations, preserving strong performance on a set of standard visual question answering tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 28 of its 41 references that are in the map. First posted on arXiv 21 Aug 2026.

Builds on

28 of 41 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery27 Jan
  2. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Models27 Dec
  3. 2024Q-VLM: Post-training Quantization for Large Vision-Language Models10 Oct

Other references · 25

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Visual Instruction Tuningcited by 7,869
  4. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  5. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  6. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,876
  7. 2025Qwen3-VL Technical Reportcited by 2,793
  8. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  9. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  10. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  11. 2025Gemma 3 Technical Reportcited by 1,824
  12. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,479
13 more
  1. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,386
  2. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  3. 2025DeepSeek-V3.2: Pushing the Frontier of Open Large Language Modelscited by 815
  4. 2016A Diagram Is Worth A Dozen Imagescited by 804
  5. 2021InfographicVQAcited by 437
  6. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 324
  7. 2022The case for 4-bit precision: k-bit Inference Scaling Lawscited by 251
  8. 2024Extreme Compression of Large Language Models via Additive Quantizationcited by 232
  9. 2023Microscaling Data Formats for Deep Learningcited by 152
  10. 2025NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cachecited by 7
  11. 2024Highly Optimized Kernels and Fine-Grained Codebooks for LLM Inference on Arm CPUscited by 5
  12. 2026Sherry: Hardware-Efficient 1.25-Bit Ternary Quantization via Fine-grained Sparsificationcited by 5
  13. 2025Optimal Formats for Weight Quantisationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023MobileVLM : A Fast, Strong and Open Vision Language Assistant for Mobile Devices
  2. 2026Empirical Recipes for Efficient and Compact Vision-Language Models
  3. 2025LiteVLM: A Low-Latency Vision-Language Model Inference Pipeline for Resource-Constrained Environments
  4. 2025SPEED-Q: Staged Processing with Enhanced Distillation towards Efficient Low-bit On-device VLM Quantization
  5. 2025Bi-VLM: Pushing Ultra-Low Precision Post-Training Quantization Boundaries in Vision-Language Models
  6. 2025Optimizing LLMs Using Quantization for Mobile Execution
  7. 2024An empirical study of LLaMA3 quantization: from LLMs to MLLMs
  8. 2025A Survey on Efficient Vision-Language Models