Quantized Convolutional Neural Networks Through the Lens of Partial Differential Equations
Not on the map. Why?
Abstract (arXiv)
Quantization of Convolutional Neural Networks (CNNs) is a common approach to ease the computational burden involved in the deployment of CNNs, especially on low-resource edge devices. However, fixed-point arithmetic is not natural to the type of computations involved in neural networks. In this work, we explore ways to improve quantized CNNs using PDE-based perspective and analysis. First, we harness the total variation (TV) approach to apply edge-aware smoothing to the feature maps throughout the network. This aims to reduce outliers in the distribution of values and promote piece-wise constant maps, which are more suitable for quantization. Secondly, we consider symmetric and stable variants of common CNNs for image classification, and Graph Convolutional Networks (GCNs) for graph node-classification. We demonstrate through several experiments that the property of forward stability preserves the action of a network under different quantization rates. As a result, stable quantized networks behave similarly to their non-quantized counterparts even though they rely on fewer parameters. We also find that at times, stability even aids in improving accuracy. These properties are of particular interest for sensitive, resource-constrained, low-power or real-time applications like autonomous driving.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 49 of its 78 references that are in the map. First posted on arXiv 31 Aug 2021.
Builds on
49 of 78 references.
References · 49
- 2015Deep Residual Learning for Image Recognitioncited by 51,809
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
- 2014Explaining and Harnessing Adversarial Examplescited by 7,287
- 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
- 2016The Cityscapes Dataset for Semantic Urban Scene Understandingcited by 4,427
- 2018Neural Ordinary Differential Equationscited by 3,733
- 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
- 2017Rethinking Atrous Convolution for Semantic Image Segmentationcited by 2,154
- 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
37 more
- 2019Searching for MobileNetV3cited by 1,984
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
- 2016Optimization Methods for Large-Scale Machine Learningcited by 1,140
- 2016Revisiting Semi-Supervised Learning with Graph Embeddingscited by 970
- 2020Simple and Deep Graph Convolutional Networkscited by 857
- 2019Learned Step Size Quantizationcited by 604
- 2020What is the State of Neural Network Pruning?cited by 578
- 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 557
- 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
- 2017Stable Architectures for Deep Neural Networkscited by 340
- 2019Data-Free Quantization Through Weight Equalization and Bias Correctioncited by 328
- 2018LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networkscited by 272
- 2021GRAND: Graph Neural Diffusioncited by 258
- 2020A Comprehensive Survey of Neural Architecture Search: Challenges and Solutionscited by 253
- 2018Deep Neural Networks Motivated by Partial Differential Equationscited by 187
- 2018Learning to Quantize Deep Networks by Optimizing Quantization Intervals with Task Losscited by 157
- 2019Improving Neural Network Quantization without Retraining using Outlier Channel Splittingcited by 150
- 2019Additive Powers-of-Two Quantization: An Efficient Non-uniform Discretization for Neural Networkscited by 143
- 2019Continuous Graph Neural Networkscited by 114
- 2021PDE-GCN: Novel Architectures for Graph Neural Networks Motivated by Partial Differential Equationscited by 106
- 2018Improving DNN Robustness to Adversarial Attacks using Jacobian Regularizationcited by 88
- 2019ANODE: Unconditionally Accurate Memory-Efficient Gradients for Neural ODEscited by 83
- 2017Adaptive Quantization for Deep Neural Networkcited by 67
- 2021Zero-shot Adversarial Quantizationcited by 44
- 2018Layer-Parallel Training of Deep Residual Neural Networkscited by 35
- 2017Deep Relaxation: partial differential equations for optimizing deep neural networkscited by 27
- 2019IMEXnet: A Forward Stable Deep Neural Networkcited by 22
- 2018Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networkscited by 20
- 2018Quantization of Fully Convolutional Networks for Accurate Biomedical Image Segmentationcited by 19
- 2020DiffGCN: Graph Convolutional Networks via Differential Operators and Algebraic Multigrid Poolingcited by 16
- 2018Forward Stability of ResNet and Its Variantscited by 12
- 2021Connections between Numerical Algorithms for PDEs and Neural Networkscited by 10
- 2019Weight Normalization based Quantization for Deep Neural Network Compressioncited by 5
- 2019LeanConvNets: Low-cost Yet Effective Convolutional Neural Networkscited by 5
- 2021Translating Numerical Concepts for PDEs into Neural Architecturescited by 4
- 2021GradFreeBits: Gradient Free Bit Allocation for Dynamic Low Precision Neural Networkscited by 2
Built on by
Citing papers per year: 2023: 1; 2024: 1; 2025: 0; 2026: 1.
3 papers.