mlsanityData to 7 Oct 2026

ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization

arXiv 2208.14286 ↗First posted 30 Aug 2022cs.LGcited by 42

Not on the map. Why?

Abstract (arXiv)

Quantization is a technique to reduce the computation and memory cost of DNN models, which are getting increasingly large. Existing quantization solutions use fixed-point integer or floating-point types, which have limited benefits, as both require more bits to maintain the accuracy of original models. On the other hand, variable-length quantization uses low-bit quantization for normal values and high-precision for a fraction of outlier values. Even though this line of work brings algorithmic benefits, it also introduces significant hardware overheads due to variable-length encoding and decoding. In this work, we propose a fixed-length adaptive numerical data type called ANT to achieve low-bit quantization with tiny hardware overheads. Our data type ANT leverages two key innovations to exploit the intra-tensor and inter-tensor adaptive opportunities in DNN models. First, we propose a particular data type, flint, that combines the advantages of float and int for adapting to the importance of different values within a tensor. Second, we propose an adaptive framework that selects the best type for each tensor according to its distribution characteristics. We design a unified processing element architecture for ANT and show its ease of integration with existing DNN accelerators. Our design results in 2.8× speedup and 2.5× energy efficiency improvement over the state-of-the-art quantization accelerators.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 53 of its 100 references that are in the map. First posted on arXiv 30 Aug 2022.

Builds on

53 of 100 references.

References · 53

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  8. 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,854
  9. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  10. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  11. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 1,998
  12. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,548
38 more
  1. 2017Mixed Precision Trainingcited by 1,135
  2. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  3. 2020Up or Down? Adaptive Rounding for Post-Training Quantizationcited by 600
  4. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
  5. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 456
  6. 2021A White Paper on Neural Network Quantizationcited by 453
  7. 2018HAQ: Hardware-Aware Automated Quantization with Mixed Precisioncited by 426
  8. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 387
  9. 2015Deep Learning with Limited Numerical Precisioncited by 368
  10. 2019Q-BERT: Hessian Based Ultra Low Precision Quantization of BERTcited by 355
  11. 2019Q8BERT: Quantized 8Bit BERTcited by 296
  12. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
  13. 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
  14. 2019HAWQ-V2: Hessian Aware trace-Weighted Quantization of Neural Networkscited by 250
  15. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 228
  16. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  17. 2020Ansor: Generating High-Performance Tensor Programs for Deep Learningcited by 199
  18. 2019Low-bit Quantization of Neural Networks for Efficient Inferencecited by 185
  19. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
  20. 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
  21. 2020GOBO: Quantizing Attention-Based NLP Models for Low Latency and Energy Efficient Inferencecited by 103
  22. 2016Convolutional Neural Networks using Logarithmic Data Representationcited by 97
  23. 2017Bit Fusion: Bit-Level Dynamically Composable Architecture for Accelerating Deep Neural Networkscited by 95
  24. 2020Rethinking Differentiable Search for Mixed-Precision Neural Networkscited by 79
  25. 2019Efficient 8-Bit Quantization of Transformer Neural Machine Language Translation Modelcited by 58
  26. 2022SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximationcited by 51
  27. 2021MQBench: Towards Reproducible and Deployable Model Quantization Benchmarkcited by 38
  28. 2020Accelerating Sparse DNN Models without Hardware-Support via Tile-Wise Sparsitycited by 30
  29. 2021DeepScaleTool : A Tool for the Accurate Estimation of Technology Scaling in the Deep-Submicron Eracited by 27
  30. 2021Dual-side Sparse Tensor Corecited by 27
  31. 2019Adversarial Defense Through Network Profiling Based Path Extractioncited by 22
  32. 2020LazyBatching: An SLA-aware Batching System for Cloud Machine Learning Inferencecited by 21
  33. 2022Transkimmer: Transformer Learns to Layer-wise Skimcited by 21
  34. 2019Effective Training of Convolutional Neural Networks with Low-bitwidth Weights and Activationscited by 16
  35. 2021Block-Skim: Efficient Question Answering for Transformercited by 16
  36. 2020Ptolemy: Architecture Support for Robust Deep Learningcited by 12
  37. 2022VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Schedulingcited by 12
  38. 2021FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Acceleratorcited by 10

50 most cited of 53 All 53 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 7; 2024: 16; 2025: 10; 2026: 8.

42 papers.

Sort:

By area:

  1. 2024LLM Inference Unveiled: Survey and Roofline Model Insightscited by 147
  2. 2023RPTQ: Reorder-based Post-training Quantization for Large Language Modelscited by 112
  3. 2024A Survey of Low-bit Large Language Models: Basics, Systems, and Algorithmscited by 30
  4. 2024Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantizationcited by 22
  5. 2024MicroScopiQ: Accelerating Foundational Models through Outlier-Aware Microscaling Quantizationcited by 22
  6. 2024A Survey: Collaborative Hardware and Software Design in the Era of Large Language Modelscited by 21
  7. 2023LUT-NN: Empower Efficient Neural Network Inference with Centroid Learning and Table Lookupcited by 19
  8. 2024Learning from Students: Applying t-Distributions to Explore Accurate and Efficient Formats for LLMscited by 18
  9. 2025Ditto: Accelerating Diffusion Model via Temporal Value Similaritycited by 18
  10. 2024Nimbus: Secure and Efficient Two-Party Inference for Transformerscited by 15
  11. 2024LUT Tensor Core: A Software-Hardware Co-Design for LUT-Based Low-Bit LLM Inferencecited by 14
  12. 2024vTensor: Flexible Virtual Tensor Management for Efficient LLM Servingcited by 13
30 more
  1. 2024BitMoD: Bit-serial Mixture-of-Datatype LLM Accelerationcited by 12
  2. 2025Low-bit Model Quantization for Deep Neural Networks: A Surveycited by 11
  3. 2025LUT-DLA: Lookup Table as Efficient Extreme Low-Bit Deep Learning Acceleratorcited by 8
  4. 2025MX+: Pushing the Limits of Microscaling Formats for Efficient Large Language Model Servingcited by 8
  5. 2024Algorithm-Hardware Co-Design of Distribution-Aware Logarithmic-Posit Encodings for Efficient DNN Inferencecited by 6
  6. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applicationscited by 6
  7. 2024Panacea: Novel DNN Accelerator using Accuracy-Preserving Asymmetric Quantization and Energy-Saving Bit-Slice Sparsitycited by 5
  8. 2025Phi: Leveraging Pattern-based Hierarchical Sparsity for High-Efficiency Spiking Neural Networkscited by 4
  9. 2023Approximate Computing Survey, Part II: Application-Specific & Architectural Approximation Techniques and Applicationscited by 3
  10. 2024BBS: Bi-directional Bit-level Sparsity for Deep Learning Accelerationcited by 3
  11. 2025P3-LLM: An Integrated NPU-PIM Accelerator for Edge LLM Inference Using Hybrid Numerical Formatscited by 3
  12. 2025Focus: A Streaming Concentration Architecture for Efficient Vision-Language Modelscited by 3
  13. 2026Adaptive Block-Scaled Data Typescited by 3
  14. 2023DyBit: Dynamic Bit-Precision Numbers for Efficient Quantized Neural Network Inferencecited by 2
  15. 2023torchdistill Meets Hugging Face Libraries for Reproducible, Coding-Free Deep Learning Studies: A Case Study on NLPcited by 2
  16. 2024Compensate Quantization Errors: Make Weights Hierarchical to Compensate Each Othercited by 2
  17. 2026EVA: Accelerating LLM Decoding via an Efficient Vector Quantization Architecturecited by 2
  18. 2022Nesting Forward Automatic Differentiation for Memory-Efficient Deep Neural Network Trainingcited by 1
  19. 2023AdaptGear: Accelerating GNN Training via Adaptive Subgraph-Level Kernels on GPUscited by 1
  20. 2023Quantized Distillation: Optimizing Driver Activity Recognition Models for Resource-Constrained Environmentscited by 1
  21. 2025ClusterFusion: Expanding Operator Fusion Scope for LLM Inference via Cluster-Level Collective Primitivecited by 1
  22. 2024Value-Driven Mixed-Precision Quantization for Patch-Based Inference on Microcontrollerscited by 0
  23. 2024Compensate Quantization Errors+: Quantized Models Are Inquisitive Learnerscited by 0
  24. 2025Hierarchical Spatial Algorithms for High-Resolution Image Quantization and Feature Extractioncited by 0
  25. 2026ELSA: An ELastic SNN Inference Architecture for Efficient Neuromorphic Computingcited by 0
  26. 2026StreamDQ: Near-Memory Weight DeQuantization in Custom HBM for Scalable AI Inference Accelerationcited by 0
  27. 2026GyRot: Leveraging Hidden Synergy between Rotation and Fine-grained Group Quantization for Low-bit LLM Inferencecited by 0
  28. 2026HBQ: Hierarchical Scaling Block Quantization with Hardware-Efficiency-Aware Design for Accurate LLM Inferencecited by 0
  29. 2026Vortex: Bridging Extreme Compression and Efficient LLM Inferencecited by 0
  30. 2026MiX: Micro-Inverted-Scaling for End-to-End Low-Bit Vision-Language Model Accelerationcited by 0