mlsanityData to 7 Oct 2026

Adaptive Block Floating-Point for Analog Deep Learning Hardware

arXiv 2205.06287 ↗First posted 12 May 2022cs.LG cs.ARcited by 4

Not on the map. Why?

Abstract (arXiv)

Analog mixed-signal (AMS) devices promise faster, more energy-efficient deep neural network (DNN) inference than their digital counterparts. However, recent studies show that DNNs on AMS devices with fixed-point numbers can incur an accuracy penalty because of precision loss. To mitigate this penalty, we present a novel AMS-compatible adaptive block floating-point (ABFP) number representation. We also introduce amplification (or gain) as a method for increasing the accuracy of the number representation without increasing the bit precision of the output. We evaluate the effectiveness of ABFP on the DNNs in the MLPerf datacenter inference benchmark -- realizing less than 1% loss in accuracy compared to FLOAT32. We also propose a novel method of finetuning for AMS devices, Differential Noise Finetuning (DNF), which samples device noise to speed up finetuning compared to conventional Quantization-Aware Training.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 23 of its 49 references that are in the map. First posted on arXiv 12 May 2022.

Builds on

23 of 49 references.

References · 23

  1. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  2. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  3. 2017Decoupled Weight Decay Regularizationcited by 1,722
  4. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
  5. 2017Mixed Precision Trainingcited by 1,135
  6. 2021A Survey of Quantization Methods for Efficient Neural Network Inferencecited by 798
  7. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 660
  8. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  9. 2020What is the State of Neural Network Pruning?cited by 578
  10. 2021Carbon Emissions and Large Neural Network Trainingcited by 561
  11. 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
  12. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
11 more
  1. 2019MLPerf Inference Benchmarkcited by 195
  2. 2020Integer Quantization for Deep Learning Inference: Principles and Empirical Evaluationcited by 183
  3. 2019What Do Compressed Deep Neural Networks Forget?cited by 164
  4. 2020Training with Quantization Noise for Extreme Model Compressioncited by 145
  5. 2019Dissecting the Graphcore IPU Architecture via Microbenchmarkingcited by 47
  6. 2018Training DNNs with Hybrid Block Floating Pointcited by 39
  7. 2018Deep Learning Inference in Facebook Data Centers: Characterization, Performance Optimizations and Hardware Implicationscited by 35
  8. 2021Dynamic Precision Analog Computing for Neural Networkscited by 12
  9. 2017Computation Error Analysis of Block Floating Point Arithmetic Oriented Convolution Neural Network Accelerator Designcited by 7
  10. 2019Mixed-Signal Charge-Domain Acceleration of Deep Neural networks through Interleaved Bit-Partitioned Arithmeticcited by 6
  11. 2017Mixed Low-precision Deep Learning Inference using Dynamic Fixed Pointcited by 4

Built on by

Cited by year

Citing papers per year: 2023: 4.

4 papers.

  1. 2023Mirage: An RNS-Based Photonic Accelerator for DNN Trainingcited by 2
  2. 2023Sensitivity-Aware Finetuning for Accuracy Recovery on Deep Learning Hardwarecited by 1
  3. 2023INT-FP-QSim: Mixed Precision and Formats For Large Language Models and Vision Transformerscited by 1
  4. 2023Photonic Accelerators for Image Segmentation in Autonomous Driving and Defect Detectioncited by 0