mlsanityData to 7 Oct 2026

MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Serving

arXiv 2510.14557 ↗First posted 16 Oct 2025cs.LG cs.ARcited by 8

On the map: Efficient LLM inference and serving → KV cache quantization

Abstract (arXiv)

Reduced-precision data formats are crucial for cost-effective serving of large language models (LLMs). While numerous reduced-precision formats have been introduced thus far, they often require intrusive modifications to the software frameworks or are rather unconventional for widespread adoption across hardware vendors. In this paper, we instead focus on recent industry-driven variants of block floating-point (BFP) formats and conduct a comprehensive analysis to push their limits for efficient LLM serving. Our analysis shows that existing ultra low-bit BFP variants struggle to provide reasonable language model performance due to outlier values in blocks. To address the outliers with BFPs, we propose MX+, a cost-effective and non-intrusive extension designed for seamless integration into the microscaling (MX) formats. MX+ builds on the key insight that the outlier does not need to use its exponent field in the element data type, which allows us to repurpose the exponent field as an extended mantissa to increase the precision of the outlier element. Our evaluation shows that MX+ achieves significantly higher model performance compared to the 4-bit MX format (MXFP4) with negligible storage overhead and slowdown, thus offering a compelling alternative to MXFP4 or MXFP6 for efficient LLM inference.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in KV cache quantization, one of 12 directions in Efficient LLM inference and serving. Builds on 32 of its 71 references that are in the map. First posted on arXiv 16 Oct 2025.

Builds on

32 of 71 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMs30 Mar

Other references · 31

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,767
  4. 2024Qwen2.5 Technical Reportcited by 4,713
  5. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  8. 2016Pointer Sentinel Mixture Modelscited by 2,992
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,338
  10. 2024Phi-4 Technical Reportcited by 675
  11. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  12. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 481
19 more
  1. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 291
  2. 2023TPU v4: An Optically Reconfigurable Supercomputer for Machine Learning with Hardware Support for Embeddingscited by 198
  3. 2023Microscaling Data Formats for Deep Learningcited by 132
  4. 2023OliVe: Accelerating Large Language Models via Hardware-friendly Outlier-Victim Pair Quantizationcited by 83
  5. 2020EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inferencecited by 69
  6. 2023LLM-FP4: 4-Bit Floating-Point Quantized Transformerscited by 65
  7. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 59
  8. 2017CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matricescited by 53
  9. 2023With Shared Microexponents, A Little Shifting Goes a Long Waycited by 47
  10. 2022ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantizationcited by 42
  11. 2018Training DNNs with Hybrid Block Floating Pointcited by 39
  12. 2021FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Roundingcited by 31
  13. 2024Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantizationcited by 22
  14. 2022Mokey: Enabling Narrow Fixed-Point Inference for Out-of-the-Box Floating-Point Transformer Modelscited by 16
  15. 2019Dissecting the NVidia Turing T4 GPU via Microbenchmarkingcited by 15
  16. 2023G10: Enabling An Efficient Unified GPU Memory and Storage Architecture with Smart Tensor Migrationscited by 10
  17. 2018Modeling Deep Learning Accelerator Enabled GPUscited by 8
  18. 2023Revisiting Block-based Quantisation: What is Important for Sub-8-bit LLM Inference?cited by 8
  19. 2024DaCapo: Accelerating Continuous Learning in Autonomous Systems for Video Analyticscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 7.

8 papers.

  1. 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimizationcited by 5
  2. 2025P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formatscited by 3
  3. 2026LATMiX: Learnable Affine Transformations for Microscaling Quantization of LLMscited by 0
  4. 2026GlowQ: Group-Shared LOw-Rank Approximation for Quantized LLMscited by 0
  5. 2026Cassandra: Enabling Reasoning LLMs at Edge via Self-Speculative Decodingcited by 0
  6. 2026A Motion-Aware Vector Quantization Framework with Centroid Reuse for Efficient VLA Inferencecited by 0
  7. 2026HBQ: Hierarchical Scaling Block Quantization with Hardware-Efficiency-Aware Design for Accurate LLM Inferencecited by 0
  8. 2026MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Accelerationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023A Dataflow Compiler for Efficient LLM Inference using Custom Microscaling Formats
  2. 2026Benchmarking Post-Training Quantization of Large Language Models under Microscaling Floating Point Formats
  3. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  4. 2024Accurate Block Quantization in LLMs with Outliers
  5. 2026Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats
  6. 2026Harmonia: Algorithm-Hardware Co-Design for Memory- and Compute-Efficient BFP-based LLM Inference ↗
  7. 2026ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs
  8. 2025NestedFP: High-Performance, Memory-Efficient Dual-Precision Floating Point Support for LLMs