# CNN Filter DB: An Empirical Investigation of Trained Convolutional Filters

arXiv: https://arxiv.org/abs/2203.15331 · HTML: https://mlsanity.com/paper/2203.15331/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 129 of its 175 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 30 of 129)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [Wide Residual Networks](https://mlsanity.com/paper/1605.07146/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Pyramid Scene Parsing Network](https://mlsanity.com/paper/1612.01105/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks](https://mlsanity.com/paper/1803.03635/) (2018)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)

## Built on by (most cited first, 20 of 20)

- [Model Zoos: A Dataset of Diverse Populations of Neural Network Models](https://mlsanity.com/paper/2209.14764/) (2022)
- [Adversarial Robustness through the Lens of Convolutional Filters](https://mlsanity.com/paper/2204.02481/) (2022)
- [An Extended Study of Human-like Behavior under Adversarial Training](https://mlsanity.com/paper/2303.12669/) (2023)
- [Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels](https://mlsanity.com/paper/2401.14469/) (2024)
- [ResiDual Transformer Alignment with Spectral Decomposition](https://mlsanity.com/paper/2411.00246/) (2024)
- [The Power of Linear Combinations: Learning with Random Convolutions](https://mlsanity.com/paper/2301.11360/) (2023)
- [Sparsified Model Zoo Twins: Investigating Populations of Sparsified Neural Network Models](https://mlsanity.com/paper/2304.13718/) (2023)
- [SecurityNet: Assessing Machine Learning Vulnerabilities on Public Models](https://mlsanity.com/paper/2310.12665/) (2023)
- [E Pluribus Unum Interpretable Convolutional Neural Networks](https://mlsanity.com/paper/2208.05369/) (2022)
- [How Do Training Methods Influence the Utilization of Vision Models?](https://mlsanity.com/paper/2410.14470/) (2024)
- [Does Medical Imaging learn different Convolution Filters?](https://mlsanity.com/paper/2210.13799/) (2022)
- [Revisiting Hidden Representations in Transfer Learning for Medical Imaging](https://mlsanity.com/paper/2302.08272/) (2023)
- [Convolutional Neural Networks Do Work with Pre-Defined Filters](https://mlsanity.com/paper/2411.18388/) (2024)
- [On the Interplay of Convolutional Padding and Adversarial Robustness](https://mlsanity.com/paper/2308.06612/) (2023)
- [Enhancing Generalization in Convolutional Neural Networks through Regularization with Edge and Line Features](https://mlsanity.com/paper/2410.16897/) (2024)
- [On the Transferability of Visual Features in Generalized Zero-Shot Learning](https://mlsanity.com/paper/2211.12494/) (2022)
- [GoogLe2Net: Going Transverse with Convolutions](https://mlsanity.com/paper/2301.00424/) (2023)
- [Searching for Effective Preprocessing Method and CNN-based Architecture with Efficient Channel Attention on Speech Emotion Recognition](https://mlsanity.com/paper/2409.04007/) (2024)
- [Towards Seamless Hierarchical Federated Learning under Intermittent Client Participation: A Stagewise Decision-Making Methodology](https://mlsanity.com/paper/2502.09303/) (2025)
- [Fuzzy-Pattern Tsetlin Machine](https://mlsanity.com/paper/2508.08350/) (2025)

## Abstract (arXiv)

Currently, many theoretical as well as practically relevant questions towards the transferability and robustness of Convolutional Neural Networks (CNNs) remain unsolved. While ongoing research efforts are engaging these problems from various angles, in most computer vision related cases these approaches can be generalized to investigations of the effects of distribution shifts in image data. In this context, we propose to study the shifts in the learned weights of trained CNN models. Here we focus on the properties of the distributions of dominantly used 3x3 convolution filter kernels. We collected and publicly provide a dataset with over 1.4 billion filters from hundreds of trained CNNs, using a wide range of datasets, architectures, and vision tasks. In a first use case of the proposed dataset, we can show highly relevant properties of many publicly available pre-trained models for practical applications: I) We analyze distribution shifts (or the lack thereof) between trained filters along different axes of meta-parameters, like visual category of the dataset, task, architecture, or layer depth. Based on these results, we conclude that model pre-training can succeed on arbitrary datasets if they meet size and variance conditions. II) We show that many pre-trained models contain degenerated filters which make them less robust and less suitable for fine-tuning on target applications. Data & Project website: https://github.com/paulgavrikov/cnn-filter-db