mlsanityData to 14 Oct 2026

All-You-Can-Fit 8-Bit Flexible Floating-Point Format for Accurate and Memory-Efficient Inference of Deep Neural Networks

arXiv 2104.07329 ↗First posted 15 Apr 2021cs.LG cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Modern deep neural network (DNN) models generally require a huge amount of weight and activation values to achieve good inference outcomes. Those data inevitably demand a massive off-chip memory capacity/bandwidth, and the situation gets even worse if they are represented in high-precision floating-point formats. Effort has been made for representing those data in different 8-bit floating-point formats, nevertheless, a notable accuracy loss is still unavoidable. In this paper we introduce an extremely flexible 8-bit floating-point (FFP8) format whose defining factors - the bit width of exponent/fraction field, the exponent bias, and even the presence of the sign bit - are all configurable. We also present a methodology to properly determine those factors so that the accuracy of model inference can be maximized. The foundation of this methodology is based on a key observation - both the maximum magnitude and the value distribution are quite dissimilar between weights and activations in most DNN models. Experimental results demonstrate that the proposed FFP8 format achieves an extremely low accuracy loss of 0.1%~ 0.3% for several representative image classification models even without the need of model retraining. Besides, it is easy to turn a classical floating-point processing unit into an FFP8-compliant one, and the extra hardware cost is minor.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 24 of its 39 references that are in the map. First posted on arXiv 15 Apr 2021.

Builds on

24 of 39 references.

References · 24

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  4. 2016Densely Connected Convolutional Networkscited by 6,720
  5. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  7. 2014Going Deeper with Convolutionscited by 4,241
  8. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,329
  10. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  11. 2017Mixed Precision Trainingcited by 1,156
  12. 2016Pruning Filters for Efficient ConvNetscited by 1,143
12 more
  1. 2017Neural Collaborative Filteringcited by 1,111
  2. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  3. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  4. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 426
  5. 2016EIE: Efficient Inference Engine on Compressed Deep Neural Networkcited by 295
  6. 2019A Study of BFLOAT16 for Deep Learning Trainingcited by 232
  7. 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 164
  8. 2018Scalable Methods for 8-bit Training of Neural Networkscited by 142
  9. 2018Mixed Precision Training of Convolutional Neural Networks using Integer Operationscited by 45
  10. 2017Flexpoint: An Adaptive Numerical Format for Efficient Training of Deep Neural Networkscited by 44
  11. 2020Shifted and Squeezed 8-bit Floating Point format for Low-Precision Training of Deep Neural Networkscited by 23
  12. 2020SmartExchange: Trading Higher-cost Memory Storage/Access for Lower-cost Computationcited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 0; 2025: 0; 2026: 1.

5 papers.

  1. 2022FP8 Quantization: The Power of the Exponentcited by 88
  2. 2023FP8 versus INT8 for efficient deep learning inferencecited by 31
  3. 2023FLIQS: One-Shot Mixed-Precision Floating-Point and Integer Quantization Searchcited by 6
  4. 2023Boolean Variation and Boolean Logic BackPropagationcited by 2
  5. 2026Compute Where it Counts: Self Optimizing Language Modelscited by 0