mlsanityData to 14 Oct 2026

GHN-QAT: Training Graph Hypernetworks to Predict Quantization-Robust Parameters of Unseen Limited Precision Neural Networks

arXiv 2309.13773 ↗First posted 24 Sep 2023cs.LG cs.AI cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Graph Hypernetworks (GHN) can predict the parameters of varying unseen CNN architectures with surprisingly good accuracy at a fraction of the cost of iterative optimization. Following these successes, preliminary research has explored the use of GHNs to predict quantization-robust parameters for 8-bit and 4-bit quantized CNNs. However, this early work leveraged full-precision float32 training and only quantized for testing. We explore the impact of quantization-aware training and/or other quantization-based training strategies on quantized robustness and performance of GHN predicted parameters for low-precision CNNs. We show that quantization-aware training can significantly improve quantized accuracy for GHN predicted parameters of 4-bit quantized CNNs and even lead to greater-than-random accuracy for 2-bit quantized CNNs. These promising results open the door for future explorations such as investigating the use of GHN predicted parameters as initialization for further quantized training of individual CNNs, further exploration of "extreme bitwidth" quantization, and mixed precision quantization schemes.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 8 of its 9 references that are in the map. First posted on arXiv 24 Sep 2023.

Builds on

8 of 9 references.

References · 8

  1. 2017Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inferencecited by 1,593
  2. 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 336
  3. 2018Graph HyperNetworks for Neural Architecture Searchcited by 132
  4. 2022Overcoming Oscillations in Quantization-Aware Trainingcited by 119
  5. 2021Parameter Prediction for Unseen Deep Architecturescited by 89
  6. 2021Do All MobileNets Quantize Poorly? Gaining Insights into the Effect of Quantization on Depthwise Separable Convolutional Networks Through the Eyes of Multi-scale Distributional Dynamicscited by 12
  7. 2020Where Should We Begin? A Low-Level Exploration of Weight Initialization Impact on Quantized Behaviour of Deep Neural Networkscited by 2
  8. 2022GHN-Q: Parameter Prediction for Unseen Quantized Convolutional Architectures via Graph Hypernetworkscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Starting Positions Matter: A Study on Better Weight Initialization for Neural Network Quantization
  2. 2023Low-bit Quantization for Deep Graph Neural Networks with Smoothness-aware Message Propagation
  3. 2024QGen: On the Ability to Generalize in Quantization Aware Training
  4. 2021Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations
  5. 2023rm A2Q: Aggregation-Aware Quantization for Graph Neural Networks
  6. 2022QFT: Post-training quantization via fast joint finetuning of all degrees of freedom
  7. 2023Tango: rethinking quantization for graph neural network training on GPUs
  8. 2023MetaGrad: Adaptive Gradient Quantization with Hypernetworks