mlsanityData to 7 Oct 2026

MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Acceleration

arXiv 2609.19683 ↗First posted 17 Sep 2026cs.AR cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Visual token pruning and efficiency

Abstract (arXiv)

The deployment of Vision-Language Models (VLMs) on edge devices is severely bottlenecked by memory bandwidth, necessitating aggressive sub-8-bit quantization. Since edge accelerators are strictly constrained by area and power, they require end-to-end quantized models. However, the extreme dynamic range gap between multi-modal tokens causes standard block formats to suffer "microscaling collapse," where a single massive outlier hijacks the shared exponent, underflowing surrounding elements and destroying attention maps. To break this bottleneck, we propose Micro-Inverted-Scaling (MiX), a novel format that mathematically inverts the microscaling paradigm: rather than grouping multiple mantissas under one shared exponent, MiX groups private, per-element exponents under a single shared mantissa. To handle asymmetric VLM outlier topologies, we introduce an adaptive dual-format (MiX-MX) inference framework. By algebraically factoring out the shared MiX mantissa, this framework maps to a custom accelerator, replacing multipliers with efficient shifters. Evaluated end-to-end on multiple VLMs, our 4.5-bit MiX formulation exhibits equivalent or superior accuracy on multi-modal benchmarks compared to NVFP4. Simultaneously, the MiX accelerator delivers a 25% improvement in area efficiency over the NVFP4 baseline and a 2.3-4.5x speedup with 1.4-2.9x energy reduction across models compared to the state-of-the-art accelerator Focus, proving the inverted-scaling datapath is physically superior for efficient VLM deployment.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 60 references that are in the map. First posted on arXiv 17 Sep 2026.

Builds on

51 of 60 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026Rethinking Practical and Efficient Quantization Calibration for Vision-Language Models8 Feb
  2. 2026VEQ: Modality-Adaptive Quantization for MoE Vision-Language Models1 Feb
  3. 2025MQuant: Unleashing the Inference Potential of Multimodal Large Language Models via Full Static Quantization1 Feb
  4. 2024MBQ: Modality-Balanced Quantization for Large Vision-Language Models27 Dec
  5. 2024Q-VLM: Post-training Quantization for Large Vision-Language Models10 Oct
  6. 2024P4Q: Learning to Prompt for Quantization in Visual-language Models26 Sep

Other references · 45

  1. 2017Attention Is All You Needcited by 59,932
  2. 2023Visual Instruction Tuningcited by 7,792
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  5. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  6. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  7. 2023MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGIcited by 2,044
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  9. 2019Towards VQA Models That Can Readcited by 1,696
  10. 2022ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoningcited by 1,449
  11. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  12. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,234
33 more
  1. 2024MiniCPM-V: A GPT-4V Level MLLM on Your Phonecited by 1,061
  2. 2022LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scalecited by 881
  3. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  4. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 805
  5. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  6. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  7. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  8. 2022ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformerscited by 517
  9. 2023OCRBench: On the Hidden Mystery of OCR in Large Multimodal Modelscited by 466
  10. 2024SpinQuant: LLM quantization with learned rotationscited by 446
  11. 2023OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Modelscited by 403
  12. 2023LLM-QAT: Data-Free Quantization Aware Training for Large Language Modelscited by 320
  13. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  14. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 158
  15. 2024SEED-Bench-2-Plus: Benchmarking Multimodal Large Language Models with Text-Rich Visual Comprehensioncited by 144
  16. 2024EfficientQAT: Efficient Quantization-Aware Training for Large Language Modelscited by 123
  17. 2020A3: Accelerating Attention Mechanisms in Neural Networks with Approximationcited by 97
  18. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 83
  19. 2022HeatViT: Hardware-Efficient Adaptive Token Pruning for Vision Transformerscited by 46
  20. 2017Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networkscited by 43
  21. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  22. 2018Training DNNs with Hybrid Block Floating Pointcited by 39
  23. 2016Bit-pragmatic Deep Neural Network Computingcited by 30
  24. 2023Ramulator 2.0: A Modern, Modular, and Extensible DRAM Simulatorcited by 26
  25. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 22
  26. 2024AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inferencecited by 17
  27. 2024BitMoD: Bit-serial Mixture-of-Datatype LLM Accelerationcited by 12
  28. 2025M-ANT: Efficient Low-bit Group Quantization for LLMs via Mathematically Adaptive Numerical Typecited by 12
  29. 2025MicroMix: Efficient Mixed-Precision Quantization with Microscaling Formats for Large Language Modelscited by 11
  30. 2025MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Servingcited by 8
  31. 2020FPRaker: A Processing Element For Accelerating Neural Network Trainingcited by 6
  32. 2025BitWave: Exploiting Column-Based Bit-Level Sparsity for Deep Learning Accelerationcited by 3
  33. 2025Focus: A Streaming Concentration Architecture for Efficient Vision-Language Modelscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026LQA: A Lightweight Quantized-Adaptive Framework for Vision-Language Models on the Edge
  2. 2026Towards Joint Quantization and Token Pruning of Vision-Language Models
  3. 2025MPQ-DMv2: Flexible Residual Mixed Precision Quantization for Low-Bit Diffusion Models with Temporal Distillation
  4. 2026Efficient Quantization-Aware Distillation with Cross-Modal Alignment for Edge Vision-Language Models
  5. 2026QMC: Efficient SLM Edge Inference via Outlier-Aware Quantization and Emergent Memories Co-Design
  6. 2026RAMP: Reinforcement Adaptive Mixed Precision Quantization for Efficient On Device LLM Inference
  7. 2026P4Q: Co-designing Token Pruning and Quantization for Vision-Language Model Acceleration
  8. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference