mlsanityData to 14 Oct 2026

Microscaling Data Formats for Deep Learning

arXiv 2310.10537 ↗First posted 16 Oct 2023cs.LG cs.AIcited by 152

Not on the map. Why?

Abstract (arXiv)

Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications. This paper evaluates Microscaling (MX) data formats that combine a per-block scaling factor with narrow floating-point and integer types for individual elements. MX formats balance the competing needs of hardware efficiency, model accuracy, and user friction. Empirical results on over two dozen benchmarks demonstrate practicality of MX data formats as a drop-in replacement for baseline FP32 for AI inference and training with low user friction. We also show the first instance of training generative language models at sub-8-bit weights, activations, and gradients with minimal accuracy loss and no modifications to the training recipe.

Key facts. Cited by 152 papers in the map (data through 14 Oct 2026). Builds on 7 of its 14 references that are in the map. First posted on arXiv 16 Oct 2023.

Builds on

7 of 14 references.

References · 7

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2019Deep Learning Recommendation Model for Personalization and Recommendation Systemscited by 327
  4. 2021VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inferencecited by 64
  5. 2023With Shared Microexponents, A Little Shifting Goes a Long Waycited by 50
  6. 2018Training DNNs with Hybrid Block Floating Pointcited by 40
  7. 2022Post-training Quantization for Neural Networks with Provable Guaranteescited by 33

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 19; 2025: 41; 2026: 91.

152 papers.

Sort:

By area:

  1. 2024DeepSeek-V3 Technical Reportcited by 811
  2. 2024QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Servingcited by 208
  3. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 170
  4. 2026Kimi K3: Open Frontier Intelligencecited by 138
  5. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Modelscited by 73
  6. 2025SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Trainingcited by 66
  7. 2025Optimizing Large Language Model Training Using FP4 Quantizationcited by 56
  8. 2024LQER: Low-Rank Quantization Error Reconstruction for LLMscited by 41
  9. 2025FP4 All the Way: Fully Quantized Training of LLMscited by 40
  10. 2025Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scalingcited by 35
  11. 2026LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generationcited by 30
  12. 2024u-μP: The Unit-Scaled Maximal Update Parametrizationcited by 29
38 more
  1. 2024Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantizationcited by 28
  2. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 28
  3. 2025TorchAO: PyTorch-Native Training-to-Serving Model Optimizationcited by 26
  4. 2026Quantization-Aware Distillation for NVFP4 Inference Accuracy Recoverycited by 22
  5. 2025Oscillation-Reduced MXFP4 Training for Vision Transformerscited by 21
  6. 2024Effective Interplay between Sparsity and Quantization: From Theory to Practicecited by 19
  7. 2024AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inferencecited by 19
  8. 2026Scalable Training of Mixture-of-Experts Models with Megatron Corecited by 19
  9. 2024LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inferencecited by 18
  10. 2025INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formatscited by 18
  11. 2025Scaling Law for Quantization-Aware Trainingcited by 17
  12. 2025M-ANT: Efficient Low-bit Group Quantization for LLMs via Mathematically Adaptive Numerical Typecited by 16
  13. 2025TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Controlcited by 14
  14. 2024Designing Efficient LLM Accelerators for Edge Devicescited by 13
  15. 2025KV Cache Transform Coding for Compact Storage in LLM Inferencecited by 13
  16. 2025BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inferencecited by 11
  17. 2025Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunitiescited by 11
  18. 2025Adaptive Prediction-Powered AutoEval with Reliability and Efficiency Guaranteescited by 11
  19. 2025Towards Fully FP8 GEMM LLM Training at Scalecited by 11
  20. 2025MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Servingcited by 11
  21. 2026HiFloat4 Format for Language Model Inferencecited by 11
  22. 2025ML-SpecQD: Multi-Level Speculative Decoding with Quantized Draftscited by 9
  23. 2025FGMP: Fine-Grained Mixed-Precision Weight and Activation Quantization for Hardware-Accelerated LLM Inferencecited by 9
  24. 2025Metis: Training LLMs with FP4 Quantizationcited by 9
  25. 2026Is Finer Better? The Limits of Microscaling Formats in Large Language Modelscited by 8
  26. 2026Full-Stack FP4: Stable LLM Pretraining with Quantized Projections, Optimizers, and Attentioncited by 8
  27. 2024Ascend HiFloat8 Format for Deep Learningcited by 7
  28. 2025Forecasting LLM Inference Performance via Hardware-Agnostic Analytical Modelingcited by 7
  29. 2026Gated Relational Alignment via Confidence-based Distillation for Efficient VLMscited by 6
  30. 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimizationcited by 6
  31. 2024FlexiBit: Fully Flexible Precision Bit-parallel Accelerator Architecture for Arbitrary Mixed Precision AIcited by 5
  32. 2025Improving Quantization with Post-Training Model Expansioncited by 5
  33. 2026ECO: Quantized Training without Full-Precision Master Weightscited by 5
  34. 2026HiFloat4 Format for Language Model Pre-training on Ascend NPUscited by 5
  35. 2026Pretraining large language models with MXFP4 on Native FP4 Hardwarecited by 5
  36. 2024Accurate Block Quantization in LLMs with Outlierscited by 4
  37. 2024Scalify: scale propagation for efficient low-precision LLM trainingcited by 4
  38. 2024OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Modelscited by 4

50 most cited of 152 All 152 (JSON)