Is In-Domain Data Really Needed? A Pilot Study on Cross-Domain Calibration for Network Quantization
Not on the map. Why?
Abstract (arXiv)
Post-training quantization methods use a set of calibration data to compute quantization ranges for network parameters and activations. The calibration data usually comes from the training dataset which could be inaccessible due to sensitivity of the data. In this work, we want to study such a problem: can we use out-of-domain data to calibrate the trained networks without knowledge of the original dataset? Specifically, we go beyond the domain of natural images to include drastically different domains such as X-ray images, satellite images and ultrasound images. We find cross-domain calibration leads to surprisingly stable performance of quantized models on 10 tasks in different image domains with 13 different calibration datasets. We also find that the performance of quantized models is correlated with the similarity of the Gram matrices between the source and calibration domains, which can be used as a criterion to choose calibration set for better performance. We believe our research opens the door to borrow cross-domain knowledge for network quantization and compression.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 30 of its 49 references that are in the map. First posted on arXiv 16 May 2021.
Builds on
30 of 49 references.
References · 30
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
- 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,845
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2014Deep Learning Face Attributes in the Wildcited by 3,679
- 2015SSD: Single Shot MultiBox Detectorcited by 3,125
- 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 829
- 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
18 more
- 2018Quantizing deep convolutional networks for efficient inference: A whitepapercited by 507
- 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
- 2016Revisiting Batch Normalization For Practical Domain Adaptationcited by 285
- 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
- 2020ZeroQ: A Novel Zero Shot Quantization Frameworkcited by 265
- 2018Bi-Real Net: Enhancing the Performance of 1-bit CNNs With Improved Representational Capability and Advanced Training Algorithmcited by 233
- 2016Trained Ternary Quantizationcited by 203
- 2016Ternary Weight Networkscited by 195
- 2015Federated Optimization:Distributed Optimization Beyond the Datacentercited by 193
- 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
- 2017Demystifying Neural Style Transfercited by 152
- 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
- 2016Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1cited by 102
- 2020Generative Low-bitwidth Data Free Quantizationcited by 87
- 2020Agriculture-Vision: A Large Aerial Image Database for Agricultural Pattern Analysiscited by 61
- 2019The Knowledge Within: Methods for Data-Free Model Compressioncited by 54
- 2019Towards Efficient Training for Neural Network Quantizationcited by 16
- 2020Multi-view Self-Constructing Graph Convolutional Networks with Adaptive Class Weighting Loss for Semantic Segmentationcited by 10
Built on by
Citing papers per year: 2022: 1; 2023: 1.
2 papers.