mlsanityData to 7 Oct 2026

Unleashing Low-Bit Inference on Ascend NPUs: A Comprehensive Evaluation of HiFloat Formats

arXiv 2602.12635 ↗First posted 13 Feb 2026cs.CL cs.AI cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

As LLMs scale, low-bit floating-point formats like MXFP and NVFP4 offer new opportunities for precision and efficiency. In this work, we evaluate HiFloat (HiF8 and HiF4), a family of formats tailored for Ascend NPUs. Through rigorous comparison across weight-activation and KV-cache tasks, we provide three key insights: (1) INT8 suits narrow-range data, while floating-point formats excel with high-variance data; (2) in 4-bit regimes, HiF4's hierarchical scaling prevents the accuracy collapse seen in integer formats; and (3) HiFloat is fully compatible with state-of-the-art post-training quantization frameworks. Overall, HiFloat provides a solution for high-efficiency LLM inference on NPUs.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 17 of its 33 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

17 of 33 references.

References · 17

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  3. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  4. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  5. 2016Pointer Sentinel Mixture Modelscited by 2,992
  6. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
  7. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,838
  8. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  9. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 604
  10. 2024QuaRot: Outlier-Free 4-Bit Inference in Rotated LLMscited by 598
  11. 2024KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantizationcited by 571
  12. 2024SpinQuant: LLM quantization with learned rotationscited by 446
5 more
  1. 2024SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Modelscited by 161
  2. 2025KVTuner: Sensitivity-Aware Layer-Wise Mixed-Precision KV Cache Quantization for Efficient and Nearly Lossless LLM Inferencecited by 36
  3. 2025Block Rotation is All You Need for MXFP4 Quantizationcited by 18
  4. 2024Ascend HiFloat8 Format for Deep Learningcited by 7
  5. 2024MixPE: Quantization and Hardware Co-design for Efficient LLM Inferencecited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026BATQuant: Outlier-resilient MXFP4 Quantization via Learnable Block-wise Optimizationcited by 5
  2. 2026QuantClaw: Precision Where It Matters for OpenClawcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2023A Dataflow Compiler for Efficient LLM Inference using Custom Microscaling Formats
  2. 2026HiFloat4 Format for Language Model Pre-training on Ascend NPUs
  3. 2026MXSens: Sensitivity-Aware Mixed-Precision Quantization for Efficient LLM Inference
  4. 2025LiquidGEMM: Hardware-Efficient W4A8 GEMM Kernel for High-Performance LLM Serving
  5. 2024Anda: Unlocking Efficient LLM Inference with a Variable-Length Grouped Activation Data Format
  6. 2026SPARQLe: Sub-Precision Activation Representation for Quantized LLM Inference ↗
  7. 2024Accurate Block Quantization in LLMs with Outliers
  8. 2026Qift: Shift-Friendly No-Zero W2 Post-Training Quantization for Rotated W2A4/KV4 LLM Inference