# Neural Network Quantization with AI Model Efficiency Toolkit (AIMET) arXiv: https://arxiv.org/abs/2201.08442 ยท HTML: https://mlsanity.com/paper/2201.08442/ ## Key facts - Cited by 24 papers in the map (data through 14 Oct 2026). - Builds on 11 of its 30 references that are in the map. - First posted on arXiv 20 Jan 2022. ## Builds on (most cited first, 11 of 11) - [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015) - [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013) - [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017) - [Learned Step Size Quantization](https://mlsanity.com/paper/1902.08153/) (2019) - [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020) - [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018) - [HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precision](https://mlsanity.com/paper/1905.03696/) (2019) - [Data-Free Quantization Through Weight Equalization and Bias Correction](https://mlsanity.com/paper/1906.04721/) (2019) - [Bayesian Bits: Unifying Quantization and Pruning](https://mlsanity.com/paper/2005.07093/) (2020) - [And the Bit Goes Down: Revisiting the Quantization of Neural Networks](https://mlsanity.com/paper/1907.05686/) (2019) - [DSConv: Efficient Convolution Operator](https://mlsanity.com/paper/1901.01928/) (2019) ## Built on by (most cited first, 24 of 24) - [FP8 versus INT8 for efficient deep learning inference](https://mlsanity.com/paper/2303.17951/) (2023) - [Machine Learning for Microcontroller-Class Hardware: A Review](https://mlsanity.com/paper/2205.14550/) (2022) - [A Practical Mixed Precision Algorithm for Post-Training Quantization](https://mlsanity.com/paper/2302.05397/) (2023) - [Jumping through Local Minima: Quantization in the Loss Landscape of Vision Transformers](https://mlsanity.com/paper/2308.10814/) (2023) - [Softmax Bias Correction for Quantized Generative Models](https://mlsanity.com/paper/2309.01729/) (2023) - [MobileNVC: Real-time 1080p Neural Video Compression on a Mobile Device](https://mlsanity.com/paper/2310.01258/) (2023) - [Quadapter: Adapter for GPT-2 Quantization](https://mlsanity.com/paper/2211.16912/) (2022) - [DIFT: Dynamic Iterative Field Transforms for Memory Efficient Optical Flow](https://mlsanity.com/paper/2306.05691/) (2023) - [Efficient Neural PDE-Solvers using Quantization Aware Training](https://mlsanity.com/paper/2308.07350/) (2023) - [EdgeRelight360: Text-Conditioned 360-Degree HDR Image Generation for Real-Time On-Device Video Portrait Relighting](https://mlsanity.com/paper/2404.09918/) (2024) - [Sub-8-bit quantization for on-device speech recognition: a regularization-free approach](https://mlsanity.com/paper/2210.09188/) (2022) - [QuickSRNet: Plain Single-Image Super-Resolution Architecture for Faster Inference on Mobile Platforms](https://mlsanity.com/paper/2303.04336/) (2023) - [Oh! We Freeze: Improving Quantized Knowledge Distillation via Signal Propagation Analysis for Large Language Models](https://mlsanity.com/paper/2403.18159/) (2024) - [EfficientBioAI: Making Bioimaging AI Models Efficient in Energy, Latency and Representation](https://mlsanity.com/paper/2306.06152/) (2023) - [SciFlow: Empowering Lightweight Optical Flow Models with Self-Cleaning Iterations](https://mlsanity.com/paper/2404.08135/) (2024) - [How to Parameterize Asymmetric Quantization Ranges for Quantization-Aware Training](https://mlsanity.com/paper/2404.16898/) (2024) - [LightCache: Memory-Efficient, Training-Free Acceleration for Video Generation](https://mlsanity.com/paper/2510.05367/) (2025) - [Adaptive Mesh-Quantization for Neural PDE Solvers](https://mlsanity.com/paper/2511.18474/) (2025) - [Torch2Chip: An End-to-end Customizable Deep Neural Network Compression and Deployment Toolkit for Prototype Hardware Accelerator Design](https://mlsanity.com/paper/2405.01775/) (2024) - [Efficient Depth Estimation for Unstable Stereo Camera Systems on AR Glasses](https://mlsanity.com/paper/2411.10013/) (2024) - [When Good Enough Is Optimal: Multiplication-Only Matrix Inversion Approximation for Quantized Gated DeltaNet](https://mlsanity.com/paper/2606.06034/) (2026) - [Speaker Diaphragm Excursion Prediction: deep attention and online adaptation](https://mlsanity.com/paper/2305.06640/) (2023) - [A Selective Quantization Tuner for ONNX Models](https://mlsanity.com/paper/2507.12196/) (2025) - [ESCA: Enabling Seamless Codec Avatar Execution through Algorithm and Hardware Co-Optimization for Virtual Reality](https://mlsanity.com/paper/2510.24787/) (2025) ## Abstract (arXiv) While neural networks have advanced the frontiers in many machine learning applications, they often come at a high computational cost. Reducing the power and latency of neural network inference is vital to integrating modern networks into edge devices with strict power and compute requirements. Neural network quantization is one of the most effective ways of achieving these savings, but the additional noise it induces can lead to accuracy degradation. In this white paper, we present an overview of neural network quantization using AI Model Efficiency Toolkit (AIMET). AIMET is a library of state-of-the-art quantization and compression algorithms designed to ease the effort required for model optimization and thus drive the broader AI ecosystem towards low latency and energy-efficient inference. AIMET provides users with the ability to simulate as well as optimize PyTorch and TensorFlow models. Specifically for quantization, AIMET includes various post-training quantization (PTQ, cf. chapter 4) and quantization-aware training (QAT, cf. chapter 5) techniques that guarantee near floating-point accuracy for 8-bit fixed-point inference. We provide a practical guide to quantization via AIMET by covering PTQ and QAT workflows, code examples and practical tips that enable users to efficiently and effectively quantize models using AIMET and reap the benefits of low-bit integer inference.