# A priori compression of convolutional neural networks for wave simulators

arXiv: https://arxiv.org/abs/2304.04964 · HTML: https://mlsanity.com/paper/2304.04964/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 11 Apr 2023.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Pruning Filters for Efficient ConvNets](https://mlsanity.com/paper/1608.08710/) (2016)
- [XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks](https://mlsanity.com/paper/1603.05279/) (2016)
- [Efficient Neural Architecture Search via Parameter Sharing](https://mlsanity.com/paper/1802.03268/) (2018)
- [BinaryConnect: Training Deep Neural Networks with binary weights during propagations](https://mlsanity.com/paper/1511.00363/) (2015)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [More Diverse Means Better: Multimodal Deep Learning Meets Remote Sensing Imagery Classification](https://mlsanity.com/paper/2008.05457/) (2020)
- [Graph Convolutional Networks for Hyperspectral Image Classification](https://mlsanity.com/paper/2008.02457/) (2020)
- [Understanding Deep Convolutional Networks](https://mlsanity.com/paper/1601.04920/) (2016)
- [Weight-Sharing Neural Architecture Search: A Battle to Shrink the Optimization Gap](https://mlsanity.com/paper/2008.01475/) (2020)
- [Convolutional Neural Network Compression through Generalized Kronecker Product Decomposition](https://mlsanity.com/paper/2109.14710/) (2021)
- [Multi-view Point Cloud Registration based on Evolutionary Multitasking with Bi-Channel Knowledge Sharing Mechanism](https://mlsanity.com/paper/2205.02996/) (2022)
- [Uncertainty quantification in a mechanical submodel driven by a Wasserstein-GAN](https://mlsanity.com/paper/2110.13680/) (2021)

## Abstract (arXiv)

Convolutional neural networks are now seeing widespread use in a variety of fields, including image classification, facial and object recognition, medical imaging analysis, and many more. In addition, there are applications such as physics-informed simulators in which accurate forecasts in real time with a minimal lag are required. The present neural network designs include millions of parameters, which makes it difficult to install such complex models on devices that have limited memory. Compression techniques might be able to resolve these issues by decreasing the size of CNN models that are created by reducing the number of parameters that contribute to the complexity of the models. We propose a compressed tensor format of convolutional layer, a priori, before the training of the neural network. 3-way kernels or 2-way kernels in convolutional layers are replaced by one-way fiters. The overfitting phenomena will be reduced also. The time needed to make predictions or time required for training using the original Convolutional Neural Networks model would be cut significantly if there were fewer parameters to deal with. In this paper we present a method of a priori compressing convolutional neural networks for finite element (FE) predictions of physical data. Afterwards we validate our a priori compressed models on physical data from a FE model solving a 2D wave equation. We show that the proposed convolutinal compression technique achieves equivalent performance as classical convolutional layers with fewer trainable parameters and lower memory footprint.