mlsanityData to 7 Oct 2026

HiFloat4 Format for Language Model Inference

arXiv 2602.11287 ↗First posted 11 Feb 2026cs.LG cs.AI cs.ARcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper introduces HiFloat4 (HiF4), a block floating-point data format tailored for deep learning. Each HiF4 unit packs 64 4-bit elements with 32 bits of shared scaling metadata, averaging 4.5 bits per value. The metadata specifies a three-level scaling hierarchy, capturing inter- and intra-group dynamic range while improving the utilization of the representational space. In addition, the large 64-element group size enables matrix multiplications to be executed in a highly fixed-point manner, significantly reducing hardware area and power consumption. To evaluate the proposed format, we conducted inference experiments on several language models, including LLaMA, Qwen, Mistral, DeepSeek-V3.1 and LongCat. Results show that HiF4 achieves higher average accuracy than the state-of-the-art NVFP4 format across multiple models and diverse downstream tasks.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 24 of its 36 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

24 of 36 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2024Qwen2.5 Technical Reportcited by 4,713
  8. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  9. 2019HellaSwag: Can a Machine Really Finish Your Sentence?cited by 4,018
  10. 2023Mistral 7Bcited by 3,065
  11. 2019PIQA: Reasoning about Physical Commonsense in Natural Languagecited by 2,818
  12. 2019BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questionscited by 2,140
12 more
  1. 2020GLU Variants Improve Transformercited by 1,724
  2. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  3. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
  4. 2016The LAMBADA dataset: Word prediction requiring a broad discourse contextcited by 960
  5. 2023A Comprehensive Overview of Large Language Modelscited by 639
  6. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  7. 2023Microscaling Data Formats for Deep Learningcited by 132
  8. 2024AI and Memory Wallcited by 89
  9. 2025Pretraining Large Language Models with NVFP4cited by 67
  10. 2025LongCat-Flash Technical Reportcited by 54
  11. 2023With Shared Microexponents, A Little Shifting Goes a Long Waycited by 47
  12. 2024Ascend HiFloat8 Format for Deep Learningcited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026HiFloat4 Format for Language Model Pre-training on Ascend NPUscited by 4
  2. 2026VFA: Relieving Vector Operations in Flash Attention with Global Maximum Pre-computationcited by 2
  3. 2026OSC: Hardware Efficient W4A4 Quantization via Outlier Separation in Channel Dimensioncited by 1
  4. 2026Collaborative Few-Step Distillation and Low-Bit Quantization for Wan2.2 Dual-Expert Video Diffusion Modelscited by 1
  5. 2026HiFA4: Training-Free 4-bit FlashAttention on Ascend HIF4 NPUs for LLM Inferencecited by 1
  6. 2026Tail-Aware HiFloat4: W4A4 Post-Training Quantization for Wan2.2cited by 0
  7. 2026SharQ: Bridging Activation Sparsity and FP4 Quantization for LLM Inferencecited by 0
  8. 2026HiFloat4 Format for End-To-End Reinforcement Learning Post-Training of Large Language Modelscited by 0
  9. 2026Activation Outliers Matter: Robust Recovery for Quantized Multimodal LLMscited by 0