# MX-SAFE: Versatile Inference- and Training-Proof Microscaling Format with On-the-Fly Exponent and Mantissa Bit Allocation

arXiv: https://arxiv.org/abs/2605.24391 · HTML: https://mlsanity.com/paper/2605.24391/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 32 references that are in the map.
- First posted on arXiv 23 May 2026.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://mlsanity.com/paper/2403.20330/) (2024)
- [SpinQuant: LLM quantization with learned rotations](https://mlsanity.com/paper/2405.16406/) (2024)
- [ResNet strikes back: An improved training procedure in timm](https://mlsanity.com/paper/2110.00476/) (2021)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://mlsanity.com/paper/2502.02737/) (2025)
- [FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization](https://mlsanity.com/paper/2303.14189/) (2023)
- [Microscaling Data Formats for Deep Learning](https://mlsanity.com/paper/2310.10537/) (2023)
- [MobileNetV4 -- Universal Models for the Mobile Ecosystem](https://mlsanity.com/paper/2404.10518/) (2024)
- [Temporal Dynamic Quantization for Diffusion Models](https://mlsanity.com/paper/2306.02316/) (2023)
- [Training LLMs with MXFP4](https://mlsanity.com/paper/2502.20586/) (2025)
- [FAST: DNN Training Under Variable Precision Block Floating Point with Stochastic Rounding](https://mlsanity.com/paper/2110.15456/) (2021)
- [Oscillation-Reduced MXFP4 Training for Vision Transformers](https://mlsanity.com/paper/2502.20853/) (2025)
- [BitMoD: Bit-serial Mixture-of-Datatype LLM Acceleration](https://mlsanity.com/paper/2411.11745/) (2024)
- [Systolic Tensor Array: An Efficient Structured-Sparse GEMM Accelerator for Mobile CNN Inference](https://mlsanity.com/paper/2005.08098/) (2020)
- [OPAL: Outlier-Preserved Microscaling Quantization Accelerator for Generative Large Language Models](https://mlsanity.com/paper/2409.05902/) (2024)

## Abstract (arXiv)

As the demand for deep learning grows, cost reduction through quantization has become essential for both training and inference. In 2022, the Open Compute Project (OCP) consortium standardized narrow precision formats for deep learning, called the microscaling (MX) format. The MX format is a hardware-friendly dynamic quantization scheme that effectively reduces the data size by sharing an 8-bit exponent across multiple operands. The MX format can be categorized into two types with their own strengths: (i) MXINT which focuses on a high precision consisting only of mantissa bits and (ii) MXFP which focuses on a wider dynamic range by allowing local exponent bits. In this work, we present a versatile MXFP format, called MX-SAFE (MXSF in short), that adaptively uses two modes, i.e., a wider mantissa mode (FP8 E2M5) and a subnormal FP mode (FP5 E3M2), to support both training and direct-cast inference. Furthermore, we propose a tile-based block design to increase hardware efficiency by reducing the burden of re-quantization process during the training with the MXSF format. Owing to the use of the proposed MXSF format, 0.05%/11.1% and 3.55%/3.57% improvements in accuracy, on average, for inference/full-training compared to MXFP8 E2M5 and MXFP8 E4M3 are observed, respectively. Moreover, we present a training-inference accelerator that supports the MXSF format and it achieves similar accuracy to the BF16 baseline while using 24.9% less total energy consumption.