# Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields

arXiv: https://arxiv.org/abs/2401.10178 · HTML: https://mlsanity.com/paper/2401.10178/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 46 references that are in the map.
- First posted on arXiv 18 Jan 2024.

## Builds on (most cited first, 24 of 24)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [Patches Are All You Need?](https://mlsanity.com/paper/2201.09792/) (2022)
- [All you need is a good init](https://mlsanity.com/paper/1511.06422/) (2015)
- [How to Start Training: The Effect of Initialization and Architecture](https://mlsanity.com/paper/1803.01719/) (2018)
- [HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutions](https://mlsanity.com/paper/2207.14284/) (2022)
- [Non-linear Convolution Filters for CNN-based Learning](https://mlsanity.com/paper/1708.07038/) (2017)
- [How to Initialize your Network? Robust Initialization for WeightNorm & ResNets](https://mlsanity.com/paper/1906.02341/) (2019)
- [Structured Receptive Fields in CNNs](https://mlsanity.com/paper/1605.02971/) (2016)
- [On-Off Center-Surround Receptive Fields for Accurate and Robust Image Classification](https://mlsanity.com/paper/2106.07091/) (2021)
- [Correspondence of Deep Neural Networks and the Brain for Visual Textures](https://mlsanity.com/paper/1806.02888/) (2018)

## Built on by (most cited first, 4 of 4)

- [Unveiling the Unseen: Identifiable Clusters in Trained Depthwise Convolutional Kernels](https://mlsanity.com/paper/2401.14469/) (2024)
- [Advancing the Biological Plausibility and Efficacy of Hebbian Convolutional Neural Networks](https://mlsanity.com/paper/2501.17266/) (2025)
- [The Master Key Filters Hypothesis: Deep Filters Are General](https://mlsanity.com/paper/2412.16751/) (2024)
- [The Quest for Universal Master Key Filters in DS-CNNs](https://mlsanity.com/paper/2509.11711/) (2025)

## Abstract (arXiv)

In this study, we present evidence suggesting that depthwise convolutional kernels are effectively replicating the structural intricacies of the biological receptive fields observed in the mammalian retina. We provide analytics of trained kernels from various state-of-the-art models substantiating this evidence. Inspired by this intriguing discovery, we propose an initialization scheme that draws inspiration from the biological receptive fields. Experimental analysis of the ImageNet dataset with multiple CNN architectures featuring depthwise convolutions reveals a marked enhancement in the accuracy of the learned model when initialized with biologically derived weights. This underlies the potential for biologically inspired computational models to further our understanding of vision processing systems and to improve the efficacy of convolutional networks.