# Wavelet Convolutions for Large Receptive Fields

arXiv: https://arxiv.org/abs/2407.05848 · HTML: https://mlsanity.com/paper/2407.05848/

## Key facts

- Cited by 74 papers in the map (data through 14 Oct 2026).
- Builds on 53 of its 70 references that are in the map.
- First posted on arXiv 8 Jul 2024.

## Builds on (most cited first, 30 of 53)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [mixup: Beyond Empirical Risk Minimization](https://mlsanity.com/paper/1710.09412/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Features](https://mlsanity.com/paper/1905.04899/) (2019)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization](https://mlsanity.com/paper/2006.16241/) (2020)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Natural Adversarial Examples](https://mlsanity.com/paper/1907.07174/) (2019)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Random Erasing Data Augmentation](https://mlsanity.com/paper/1708.04896/) (2017)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Learning Robust Global Representations by Penalizing Local Predictive Power](https://mlsanity.com/paper/1905.13549/) (2019)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Intriguing Properties of Vision Transformers](https://mlsanity.com/paper/2105.10497/) (2021)
- [Cascade R-CNN: High Quality Object Detection and Instance Segmentation](https://mlsanity.com/paper/1906.09756/) (2019)
- [GhostNet: More Features from Cheap Operations](https://mlsanity.com/paper/1911.11907/) (2019)

## Built on by (most cited first, 30 of 74)

- [WF-VAE: Enhancing Video VAE by Wavelet-Driven Energy Flow for Latent Video Diffusion Model](https://mlsanity.com/paper/2411.17459/) (2024)
- [MobileMamba: Lightweight Multi-Receptive Visual Mamba Network](https://mlsanity.com/paper/2411.15941/) (2024)
- [DiC: Rethinking Conv3x3 Designs in Diffusion Models](https://mlsanity.com/paper/2501.00603/) (2024)
- [WaveMamba: Wavelet-Driven Mamba Fusion for RGB-Infrared Object Detection](https://mlsanity.com/paper/2507.18173/) (2025)
- [CWNet: Causal Wavelet Network for Low-Light Image Enhancement](https://mlsanity.com/paper/2507.10689/) (2025)
- [Activation Space Selectable Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2408.08338/) (2024)
- [Light of Normals: Unified Feature Representation for Universal Photometric Stereo](https://mlsanity.com/paper/2506.18882/) (2025)
- [Controllable-LPMoE: Adapting to Challenging Object Segmentation via Dynamic Local Priors from Mixture-of-Experts](https://mlsanity.com/paper/2510.21114/) (2025)
- [FSDENet: A Frequency and Spatial Domains based Detail Enhancement Network for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2510.00059/) (2025)
- [Real Noise Decoupling for Hyperspectral Image Denoising](https://mlsanity.com/paper/2511.17196/) (2025)
- [Improving the Effective Receptive Field of Message-Passing Neural Networks](https://mlsanity.com/paper/2505.23185/) (2025)
- [DWTGS: Rethinking Frequency Regularization for Sparse-view 3D Gaussian Splatting](https://mlsanity.com/paper/2507.15690/) (2025)
- [3D Wavelet Convolutions with Extended Receptive Fields for Hyperspectral Image Classification](https://mlsanity.com/paper/2504.10795/) (2025)
- [Learning Adaptive Lighting via Channel-Aware Guidance](https://mlsanity.com/paper/2412.01493/) (2024)
- [Dynamic High-frequency Convolution for Infrared Small Target Detection](https://mlsanity.com/paper/2602.02969/) (2026)
- [OCTAMamba: A State-Space Model Approach for Precision OCTA Vasculature Segmentation](https://mlsanity.com/paper/2409.08000/) (2024)
- [WaveNet-SF: A Hybrid Network for Retinal Disease Detection Based on Wavelet Transform in Spatial-Frequency Domain](https://mlsanity.com/paper/2501.11854/) (2025)
- [Cross-Frequency Implicit Neural Representation with Self-Evolving Parameters](https://mlsanity.com/paper/2504.10929/) (2025)
- [FreqU-FNet: Frequency-Aware U-Net for Imbalanced Medical Image Segmentation](https://mlsanity.com/paper/2505.17544/) (2025)
- [WaveFormer: A Lightweight Transformer Model for sEMG-based Gesture Recognition](https://mlsanity.com/paper/2506.11168/) (2025)
- [SPJFNet: Self-Mining Prior-Guided Joint Frequency Enhancement for Ultra-Efficient Dark Image Restoration](https://mlsanity.com/paper/2508.04041/) (2025)
- [UniConvNet: Expanding Effective Receptive Field while Maintaining Asymptotically Gaussian Distribution for ConvNets of Any Scale](https://mlsanity.com/paper/2508.09000/) (2025)
- [AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception](https://mlsanity.com/paper/2508.16881/) (2025)
- [Advancing 3D Gaussian Splatting Editing with Complementary and Consensus Information](https://mlsanity.com/paper/2503.11601/) (2025)
- [TimePoint: Accelerated Time Series Alignment via Self-Supervised Keypoint and Descriptor Learning](https://mlsanity.com/paper/2505.23475/) (2025)
- [BeltCrack: the First Sequential-image Industrial Conveyor Belt Crack Detection Dataset and Its Baseline with Triple-domain Feature Learning](https://mlsanity.com/paper/2506.17892/) (2025)
- [SG-DeepONet: Source-generalized deep operator learning for full waveform inversion](https://mlsanity.com/paper/2408.08005/) (2024)
- [Frequency Domain Enhanced U-Net for Low-Frequency Information-Rich Image Segmentation in Surgical and Deep-Sea Exploration Robots](https://mlsanity.com/paper/2502.03829/) (2025)
- [DSU-Net:An Improved U-Net Model Based on DINOv2 and SAM2 with Multi-scale Cross-model Feature Enhancement](https://mlsanity.com/paper/2503.21187/) (2025)
- [PRAD: Periapical Radiograph Analysis Dataset and Benchmark Model Development](https://mlsanity.com/paper/2504.07760/) (2025)

## Abstract (arXiv)

In recent years, there have been attempts to increase the kernel size of Convolutional Neural Nets (CNNs) to mimic the global receptive field of Vision Transformers' (ViTs) self-attention blocks. That approach, however, quickly hit an upper bound and saturated way before achieving a global receptive field. In this work, we demonstrate that by leveraging the Wavelet Transform (WT), it is, in fact, possible to obtain very large receptive fields without suffering from over-parameterization, e.g., for a k × k receptive field, the number of trainable parameters in the proposed method grows only logarithmically with k. The proposed layer, named WTConv, can be used as a drop-in replacement in existing architectures, results in an effective multi-frequency response, and scales gracefully with the size of the receptive field. We demonstrate the effectiveness of the WTConv layer within ConvNeXt and MobileNetV2 architectures for image classification, as well as backbones for downstream tasks, and show it yields additional properties such as robustness to image corruption and an increased response to shapes over textures. Our code is available at https://github.com/BGU-CS-VIL/WTConv.