# On filter design in deep convolutional neural network

arXiv: https://arxiv.org/abs/2410.21644 · HTML: https://mlsanity.com/paper/2410.21644/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 31 of its 74 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [Context Encoders: Feature Learning by Inpainting](https://mlsanity.com/paper/1604.07379/) (2016)
- [Deep Clustering for Unsupervised Learning of Visual Features](https://mlsanity.com/paper/1807.05520/) (2018)
- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Resnet in Resnet: Generalizing Residual Architectures](https://mlsanity.com/paper/1603.08029/) (2016)
- [On weight initialization in deep neural networks](https://mlsanity.com/paper/1704.08863/) (2017)
- [Interpretable Convolutional Neural Networks via Feedforward Design](https://mlsanity.com/paper/1810.02786/) (2018)
- [PixelHop++: A Small Successive-Subspace-Learning-Based (SSL-based) Model for Image Classification](https://mlsanity.com/paper/2002.03141/) (2020)
- [Understanding Convolutional Neural Networks with A Mathematical Model](https://mlsanity.com/paper/1609.04112/) (2016)
- [Comparing Different Deep Learning Architectures for Classification of Chest Radiographs](https://mlsanity.com/paper/2002.08991/) (2020)
- [On Data-Driven Saak Transform](https://mlsanity.com/paper/1710.04176/) (2017)
- [E-PixelHop: An Enhanced PixelHop Method for Object Classification](https://mlsanity.com/paper/2107.02966/) (2021)
- [Weight Initialization Techniques for Deep Learning Algorithms in Remote Sensing: Recent Trends and Future Perspectives](https://mlsanity.com/paper/2102.07004/) (2021)
- [An Image Clustering Auto-Encoder Based on Predefined Evenly-Distributed Class Centroids and MMD Distance](https://mlsanity.com/paper/1906.03905/) (2019)
- [Learning to ignore: rethinking attention in CNNs](https://mlsanity.com/paper/2111.05684/) (2021)
- [Dendritic Self-Organizing Maps for Continual Learning](https://mlsanity.com/paper/2110.13611/) (2021)
- [CSNNs: Unsupervised, Backpropagation-free Convolutional Neural Networks for Representation Learning](https://mlsanity.com/paper/2001.10388/) (2020)
- [PixelHop: A Successive Subspace Learning (SSL) Method for Object Classification](https://mlsanity.com/paper/1909.08190/) (2019)
- [Teacher-Class Network: A Neural Network Compression Mechanism](https://mlsanity.com/paper/2004.03281/) (2020)
- [Beyond Classification: Knowledge Distillation using Multi-Object Impressions](https://mlsanity.com/paper/2110.14215/) (2021)
- [A SOM-based Gradient-Free Deep Learning Method with Convergence Analysis](https://mlsanity.com/paper/2101.05612/) (2021)

## Abstract (arXiv)

The deep convolutional neural network (DCNN) in computer vision has given promising results. It is widely applied in many areas, from medicine, agriculture, self-driving car, biometric system, and almost all computer vision-based applications. Filters or weights are the critical elements responsible for learning in DCNN. Backpropagation has been the primary learning algorithm for DCNN and provides promising results, but the size and numbers of the filters remain hyper-parameters. Various studies have been done in the last decade on semi-supervised, self-supervised, and unsupervised methods and their properties. The effects of filter initialization, size-shape selection, and the number of filters on learning and optimization have not been investigated in a separate publication to collate all the options. Such attributes are often treated as hyper-parameters and lack mathematical understanding. Computer vision algorithms have many limitations in real-life applications, and understanding the learning process is essential to have some significant improvement. To the best of our knowledge, no separate investigation has been published discussing the filters; this is our primary motivation. This study focuses on arguments for choosing specific physical parameters of filters, initialization, and learning technic over scattered methods. The promising unsupervised approaches have been evaluated. Additionally, the limitations, current challenges, and future scope have been discussed in this paper.