mlsanityData to 14 Oct 2026

Regularization-based Framework for Quantization-, Fault- and Variability-Aware Training

arXiv 2503.01297 ↗First posted 3 Mar 2025cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

Efficient inference is critical for deploying deep learning models on edge AI devices. Low-bit quantization (e.g., 3- and 4-bit) with fixed-point arithmetic improves efficiency, while low-power memory technologies like analog nonvolatile memory enable further gains. However, these methods introduce non-ideal hardware behavior, including bit faults and device-to-device variability. We propose a regularization-based quantization-aware training (QAT) framework that supports fixed, learnable step-size, and learnable non-uniform quantization, achieving competitive results on CIFAR-10 and ImageNet. Our method also extends to Spiking Neural Networks (SNNs), demonstrating strong performance on 4-bit networks on CIFAR10-DVS and N-Caltech 101. Beyond quantization, our framework enables fault and variability-aware fine-tuning, mitigating stuck-at faults (fixed weight bits) and device resistance variability. Compared to prior fault-aware training, our approach significantly improves performance recovery under upto 20% bit-fault rate and 40% device-to-device variability. Our results establish a generalizable framework for quantization and robustness-aware training, enhancing efficiency and reliability in low-power, non-ideal hardware.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 32 of its 60 references that are in the map. First posted on arXiv 3 Mar 2025.

Builds on

32 of 60 references.

References · 32

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,498
  6. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  7. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  8. 2022GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformerscited by 1,919
  9. 2019Learned Step Size Quantizationcited by 628
  10. 2019Surrogate Gradient Learning in Spiking Neural Networkscited by 626
  11. 2018PACT: Parameterized Clipping Activation for Quantized Neural Networkscited by 467
  12. 2020Incorporating Learnable Membrane Time Constant to Enhance Learning of Spiking Neural Networkscited by 345
20 more
  1. 2015Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccadescited by 314
  2. 2020Going Deeper With Directly-Trained Larger Spiking Neural Networkscited by 278
  3. 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 274
  4. 2023SpikingJelly: An open-source machine learning infrastructure platform for spike-based intelligencecited by 169
  5. 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 160
  6. 2017Apprentice: Using Knowledge Distillation Techniques To Improve Low-Precision Network Accuracycited by 101
  7. 2022Mixed-Precision Neural Network Quantization via Learned Layer-wise Importancecited by 54
  8. 2022Neuromorphic Data Augmentation for Training Spiking Neural Networkscited by 52
  9. 2023Optimize Weight Rounding via Signed Gradient Descent for the Quantization of LLMscited by 50
  10. 2021Learnable Companding Quantization for Accurate Low-bit Neural Networkscited by 42
  11. 2021A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrayscited by 39
  12. 2015Resiliency of Deep Neural Networks under Quantizationcited by 35
  13. 2018Discovering Low-Precision Networks Close to Full-Precision Networks for Efficient Embedded Inferencecited by 33
  14. 2020DNN+NeuroSim V2.0: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators for On-chip Trainingcited by 33
  15. 2019EDEN: Enabling Energy-Efficient, High-Performance Deep Neural Network Inference Using Approximate DRAMcited by 24
  16. 2020FAT: Training Neural Networks for Reliable Inference Under Hardware Faultscited by 15
  17. 2019Training on the Edge: The why and the howcited by 9
  18. 2019SinReQ: Generalized Sinusoidal Regularization for Low-Bitwidth Deep Quantized Trainingcited by 2
  19. 2022Exploiting the Partly Scratch-off Lottery Ticket for Quantization-Aware Trainingcited by 2
  20. 2023FAQ: Mitigating the Impact of Faults in the Weight Memory of DNN Accelerators through Fault-Aware Quantizationcited by 2

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flashcited by 4

Similar, not cited

Close in wording, no citation either way

  1. 2023On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks
  2. 2024Memory Faults in Activation-sparse Quantized Deep Neural Networks: Analysis and Mitigation using Sharpness-aware Training
  3. 2021Variability-Aware Training and Self-Tuning of Highly Quantized DNNs for Analog PIM
  4. 2026Scale When Needed: Adaptive Neuron-level Mixed Precision Quantization Aware Training
  5. 2022PIM-QAT: Neural Network Quantization for Processing-In-Memory (PIM) Systems
  6. 2022Edge Inference with Fully Differentiable Quantized Mixed Precision Neural Networks
  7. 2025QS4D: Quantization-aware training for efficient hardware deployment of structured state-space sequential models
  8. 2024QGen: On the Ability to Generalize in Quantization Aware Training