mlsanityData to 14 Oct 2026

On Calibration of Modern Quantized Efficient Neural Networks

arXiv 2309.13866 ↗First posted 25 Sep 2023cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

We explore calibration properties at various precisions for three architectures: ShuffleNetv2, GhostNet-VGG, and MobileOne; and two datasets: CIFAR-100 and PathMNIST. The quality of calibration is observed to track the quantization quality; it is well-documented that performance worsens with lower precision, and we observe a similar correlation with poorer calibration. This becomes especially egregious at 4-bit activation regime. GhostNet-VGG is shown to be the most robust to overall performance drop at lower precision. We find that temperature scaling can improve calibration error for quantized networks, with some caveats. We hope that these preliminary insights can lead to more opportunities for explainable and reliable EdgeML.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 29 references that are in the map. First posted on arXiv 25 Sep 2023.

Builds on

21 of 29 references.

References · 21

  1. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  2. 2017On Calibration of Modern Neural Networkscited by 4,960
  3. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,611
  4. 2019Searching for MobileNetV3cited by 1,994
  5. 2019Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shiftcited by 1,287
  6. 2019When Does Label Smoothing Help?cited by 988
  7. 2021MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classificationcited by 690
  8. 2021A White Paper on Neural Network Quantizationcited by 472
  9. 2019GhostNet: More Features from Cheap Operationscited by 410
  10. 2019HAWQ: Hessian AWare Quantization of Neural Networks with Mixed-Precisioncited by 402
  11. 2021Revisiting the Calibration of Modern Neural Networkscited by 390
  12. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 217
9 more
  1. 2021Differentiable Model Compression via Pseudo Quantization Noisecited by 45
  2. 2020HMQ: Hardware Friendly Mixed Precision Quantization Block for CNNscited by 43
  3. 2019Well-calibrated Model Uncertainty with Temperature Scaling for Dropout Variational Inferencecited by 36
  4. 2022Beyond calibration: estimating the grouping loss of modern neural networkscited by 32
  5. 2021Sharpness-aware Quantization for Deep Neural Networkscited by 25
  6. 2021Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamicscited by 12
  7. 2023Towards unraveling calibration biases in medical image analysiscited by 8
  8. 2021An Underexplored Dilemma between Confidence and Calibration in Quantized Neural Networkscited by 2
  9. 2022AugShuffleNet: Communicate More, Compute Lesscited by 1

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Hierarchical Selective Classificationcited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2023A Practical Mixed Precision Algorithm for Post-Training Quantization
  2. 2022AMED: Automatic Mixed-Precision Quantization for Edge Devices
  3. 2026RAMP: Robust Adaptive Mixed-Precision Quantization for Edge CPU Vision Models
  4. 2023On-Chip Hardware-Aware Quantization for Mixed Precision Neural Networks
  5. 2021Data-free mixed-precision quantization using novel sensitivity metric
  6. 2023Free Bits: Latency Optimization of Mixed-Precision Quantized Neural Networks on the Edge
  7. 2022Edge Inference with Fully Differentiable Quantized Mixed Precision Neural Networks
  8. 2021Confounding Tradeoffs for Neural Network Quantization