# Microscaling Data Formats for Deep Learning

arXiv: https://arxiv.org/abs/2310.10537 · HTML: https://mlsanity.com/paper/2310.10537/

## Key facts

- Cited by 152 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 14 references that are in the map.
- First posted on arXiv 16 Oct 2023.

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [Deep Learning Recommendation Model for Personalization and Recommendation Systems](https://mlsanity.com/paper/1906.00091/) (2019)
- [VS-Quant: Per-vector Scaled Quantization for Accurate Low-Precision Neural Network Inference](https://mlsanity.com/paper/2102.04503/) (2021)
- [With Shared Microexponents, A Little Shifting Goes a Long Way](https://mlsanity.com/paper/2302.08007/) (2023)
- [Training DNNs with Hybrid Block Floating Point](https://mlsanity.com/paper/1804.01526/) (2018)
- [Post-training Quantization for Neural Networks with Provable Guarantees](https://mlsanity.com/paper/2201.11113/) (2022)

## Built on by (most cited first, 30 of 152)

- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving](https://mlsanity.com/paper/2405.04532/) (2024)
- [SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models](https://mlsanity.com/paper/2411.05007/) (2024)
- [Kimi K3: Open Frontier Intelligence](https://mlsanity.com/paper/2607.24653/) (2026)
- [Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models](https://mlsanity.com/paper/2504.04823/) (2025)
- [SageAttention3: Microscaling FP4 Attention for Inference and An Exploration of 8-Bit Training](https://mlsanity.com/paper/2505.11594/) (2025)
- [Optimizing Large Language Model Training Using FP4 Quantization](https://mlsanity.com/paper/2501.17116/) (2025)
- [LQER: Low-Rank Quantization Error Reconstruction for LLMs](https://mlsanity.com/paper/2402.02446/) (2024)
- [FP4 All the Way: Fully Quantized Training of LLMs](https://mlsanity.com/paper/2505.19115/) (2025)
- [Four Over Six: More Accurate NVFP4 Quantization with Adaptive Block Scaling](https://mlsanity.com/paper/2512.02010/) (2025)
- [LongLive-2.0: An NVFP4 Parallel Infrastructure for Long Video Generation](https://mlsanity.com/paper/2605.18739/) (2026)
- [u-μP: The Unit-Scaled Maximal Update Parametrization](https://mlsanity.com/paper/2407.17465/) (2024)
- [Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization](https://mlsanity.com/paper/2406.12930/) (2024)
- [MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantization](https://mlsanity.com/paper/2411.05282/) (2024)
- [TorchAO: PyTorch-Native Training-to-Serving Model Optimization](https://mlsanity.com/paper/2507.16099/) (2025)
- [Quantization-Aware Distillation for NVFP4 Inference Accuracy Recovery](https://mlsanity.com/paper/2601.20088/) (2026)
- [Oscillation-Reduced MXFP4 Training for Vision Transformers](https://mlsanity.com/paper/2502.20853/) (2025)
- [Effective Interplay between Sparsity and Quantization: From Theory to Practice](https://mlsanity.com/paper/2405.20935/) (2024)
- [AMXFP4: Taming Activation Outliers with Asymmetric Microscaling Floating-Point for 4-bit LLM Inference](https://mlsanity.com/paper/2411.09909/) (2024)
- [Scalable Training of Mixture-of-Experts Models with Megatron Core](https://mlsanity.com/paper/2603.07685/) (2026)
- [LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inference](https://mlsanity.com/paper/2408.06003/) (2024)
- [INT v.s. FP: A Comprehensive Study of Fine-Grained Low-bit Quantization Formats](https://mlsanity.com/paper/2510.25602/) (2025)
- [Scaling Law for Quantization-Aware Training](https://mlsanity.com/paper/2505.14302/) (2025)
- [M-ANT: Efficient Low-bit Group Quantization for LLMs via Mathematically Adaptive Numerical Type](https://mlsanity.com/paper/2502.18755/) (2025)
- [TetraJet-v2: Accurate NVFP4 Training for Large Language Models with Oscillation Suppression and Outlier Control](https://mlsanity.com/paper/2510.27527/) (2025)
- [Designing Efficient LLM Accelerators for Edge Devices](https://mlsanity.com/paper/2408.00462/) (2024)
- [KV Cache Transform Coding for Compact Storage in LLM Inference](https://mlsanity.com/paper/2511.01815/) (2025)
- [BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference](https://mlsanity.com/paper/2501.01144/) (2025)
- [Low-Precision Training of Large Language Models: Methods, Challenges, and Opportunities](https://mlsanity.com/paper/2505.01043/) (2025)
- [Adaptive Prediction-Powered AutoEval with Reliability and Efficiency Guarantees](https://mlsanity.com/paper/2505.18659/) (2025)

## Abstract (arXiv)

Narrow bit-width data formats are key to reducing the computational and storage costs of modern deep learning applications. This paper evaluates Microscaling (MX) data formats that combine a per-block scaling factor with narrow floating-point and integer types for individual elements. MX formats balance the competing needs of hardware efficiency, model accuracy, and user friction. Empirical results on over two dozen benchmarks demonstrate practicality of MX data formats as a drop-in replacement for baseline FP32 for AI inference and training with low user friction. We also show the first instance of training generative language models at sub-8-bit weights, activations, and gradients with minimal accuracy loss and no modifications to the training recipe.