# Structured Receptive Fields in CNNs

arXiv: https://arxiv.org/abs/1605.02971 · HTML: https://mlsanity.com/paper/1605.02971/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2016.

## Built on by (most cited first, 30 of 32)

- [FlexConv: Continuous Kernel Convolutions with Differentiable Kernel Sizes](https://mlsanity.com/paper/2110.08059/) (2021)
- [Resolution learning in deep convolutional networks using scale-space theory](https://mlsanity.com/paper/2106.03412/) (2021)
- [Dilated convolution with learnable spacings](https://mlsanity.com/paper/2112.03740/) (2021)
- [Parametric Scattering Networks](https://mlsanity.com/paper/2107.09539/) (2021)
- [On-Off Center-Surround Receptive Fields for Accurate and Robust Image Classification](https://mlsanity.com/paper/2106.07091/) (2021)
- [Spectral Leakage and Rethinking the Kernel Size in CNNs](https://mlsanity.com/paper/2101.10143/) (2021)
- [Covariance properties under natural image transformations for the generalized Gaussian derivative model for visual receptive fields](https://mlsanity.com/paper/2303.09803/) (2023)
- [Riesz networks: scale invariant neural networks in a single forward pass](https://mlsanity.com/paper/2305.04665/) (2023)
- [Foveation in the Era of Deep Learning](https://mlsanity.com/paper/2312.01450/) (2023)
- [Discrete approximations of Gaussian smoothing and Gaussian derivatives](https://mlsanity.com/paper/2311.11317/) (2023)
- [Equivariant Wavelets: Fast Rotation and Translation Invariant Wavelet Scattering Transforms](https://mlsanity.com/paper/2104.11244/) (2021)
- [Frequency learning for structured CNN filters with Gaussian fractional derivatives](https://mlsanity.com/paper/2111.06660/) (2021)
- [Moving Frame Net: SE(3)-Equivariant Network for Volumes](https://mlsanity.com/paper/2211.03420/) (2022)
- [Dilated Convolution with Learnable Spacings: beyond bilinear interpolation](https://mlsanity.com/paper/2306.00817/) (2023)
- [Neural Echos: Depthwise Convolutional Filters Replicate Biological Receptive Fields](https://mlsanity.com/paper/2401.10178/) (2024)
- [Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators](https://mlsanity.com/paper/2405.05095/) (2024)
- [Covariant spatio-temporal receptive fields for spiking neural networks](https://mlsanity.com/paper/2405.00318/) (2024)
- [Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations](https://mlsanity.com/paper/2409.11140/) (2024)
- [Symmetry Structured Convolutional Neural Networks](https://mlsanity.com/paper/2203.02056/) (2022)
- [Deep Continuous Networks](https://mlsanity.com/paper/2402.01557/) (2024)
- [3D Lymphoma Segmentation on PET/CT Images via Multi-Scale Information Fusion with Cross-Attention](https://mlsanity.com/paper/2402.02349/) (2024)
- [PDE-CNNs: Axiomatic Derivations and Applications](https://mlsanity.com/paper/2403.15182/) (2024)
- [Automatic Prediction of Stroke Treatment Outcomes: Latest Advances and Perspectives](https://mlsanity.com/paper/2412.04812/) (2024)
- [FOVI: A biologically-inspired foveated interface for deep vision models](https://mlsanity.com/paper/2602.03766/) (2026)
- [Blind Image Inpainting with Sparse Directional Filter Dictionaries for Lightweight CNNs](https://mlsanity.com/paper/2205.06597/) (2022)
- [Scaling Continuous Kernels with Sparse Fourier Domain Learning](https://mlsanity.com/paper/2409.09875/) (2024)
- [Evaluating Graphical Perception Capabilities of Vision Transformers](https://mlsanity.com/paper/2602.18178/) (2026)
- [Built-in Elastic Transformations for Improved Robustness](https://mlsanity.com/paper/2107.09391/) (2021)
- [Wiggling Weights to Improve the Robustness of Classifiers](https://mlsanity.com/paper/2111.09779/) (2021)
- [Approximating Continuous Convolutions for Deep Network Compression](https://mlsanity.com/paper/2210.08951/) (2022)

## Abstract (arXiv)

Learning powerful feature representations with CNNs is hard when training data are limited. Pre-training is one way to overcome this, but it requires large datasets sufficiently similar to the target domain. Another option is to design priors into the model, which can range from tuned hyperparameters to fully engineered representations like Scattering Networks. We combine these ideas into structured receptive field networks, a model which has a fixed filter basis and yet retains the flexibility of CNNs. This flexibility is achieved by expressing receptive fields in CNNs as a weighted sum over a fixed basis which is similar in spirit to Scattering Networks. The key difference is that we learn arbitrary effective filter sets from the basis rather than modeling the filters. This approach explicitly connects classical multiscale image analysis with general CNNs. With structured receptive field networks, we improve considerably over unstructured CNNs for small and medium dataset scenarios as well as over Scattering for large datasets. We validate our findings on ILSVRC2012, Cifar-10, Cifar-100 and MNIST. As a realistic small dataset example, we show state-of-the-art classification results on popular 3D MRI brain-disease datasets where pre-training is difficult due to a lack of large public datasets in a similar domain.