# Nanoscaling Floating-Point (NxFP): NanoMantissa, Adaptive Microexponents, and Code Recycling for Direct-Cast Compression of Large Language Models

arXiv: https://arxiv.org/abs/2412.19821 · HTML: https://mlsanity.com/paper/2412.19821/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 34 references that are in the map.
- First posted on arXiv 15 Dec 2024.

## Builds on (most cited first, 3 of 3)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)

## Built on by (most cited first, 7 of 7)

- [BlockDialect: Block-wise Fine-grained Mixed Format Quantization for Energy-Efficient LLM Inference](https://mlsanity.com/paper/2501.01144/) (2025)
- [ML-SpecQD: Multi-Level Speculative Decoding with Quantized Drafts](https://mlsanity.com/paper/2503.13565/) (2025)
- [Is Finer Better? The Limits of Microscaling Formats in Large Language Models](https://mlsanity.com/paper/2601.19026/) (2026)
- [RPU -- A Reasoning Processing Unit](https://mlsanity.com/paper/2602.18568/) (2026)
- [LOCALUT: Harnessing Capacity-Computation Tradeoffs for LUT-Based Inference in DRAM-PIM](https://mlsanity.com/paper/2604.04523/) (2026)
- [SemanticDialect: Semantic-Aware Mixed-Format Quantization for Video Diffusion Transformers](https://mlsanity.com/paper/2603.02883/) (2026)
- [Heterogeneity-Aware Microscaling for Efficient Low-Bit LLM Inference](https://mlsanity.com/paper/2608.03867/) (2026)

## Abstract (arXiv)

As cutting-edge large language models (LLMs) continue to transform various industries, their fast-growing model size and sequence length have led to memory traffic and capacity challenges. Recently, AMD, Arm, Intel, Meta, Microsoft, NVIDIA, and Qualcomm have proposed a Microscaling standard (Mx), which augments block floating-point with microexponents to achieve promising perplexity-to-footprint trade-offs. However, the Microscaling suffers from significant perplexity degradation on modern LLMs with less than six bits. This paper profiles modern LLMs and identifies three main challenges of low-bit Microscaling format, i.e., inaccurate tracking of outliers, vacant quantization levels, and wasted binary code. In response, Nanoscaling (NxFP) proposes three techniques, i.e., NanoMantissa, Adaptive Microexponent, and Code Recycling to enable better accuracy and smaller memory footprint than state-of-the-art MxFP. Experimental results on direct-cast inference across various modern LLMs demonstrate that our proposed methods outperform state-of-the-art MxFP by up to 0.64 in perplexity and by up to 30% in accuracy on MMLU benchmarks. Furthermore, NxFP reduces memory footprint by up to 16% while achieving comparable perplexity as MxFP.