# Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution

arXiv: https://arxiv.org/abs/1904.05049 · HTML: https://mlsanity.com/paper/1904.05049/

## Key facts

- Cited by 140 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2019.

## Built on by (most cited first, 30 of 140)

- [Is Space-Time Attention All You Need for Video Understanding?](https://mlsanity.com/paper/2102.05095/) (2021)
- [Multiscale Vision Transformers](https://mlsanity.com/paper/2104.11227/) (2021)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [MViTv2: Improved Multiscale Vision Transformers for Classification and Detection](https://mlsanity.com/paper/2112.01526/) (2021)
- [VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text](https://mlsanity.com/paper/2104.11178/) (2021)
- [PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition](https://mlsanity.com/paper/2403.17695/) (2024)
- [Run, Don't Walk: Chasing Higher FLOPS for Faster Neural Networks](https://mlsanity.com/paper/2303.03667/) (2023)
- [Inception Transformer](https://mlsanity.com/paper/2205.12956/) (2022)
- [Selective-Stereo: Adaptive Frequency Information Selection for Stereo Matching](https://mlsanity.com/paper/2403.00486/) (2024)
- [Diverse Branch Block: Building a Convolution as an Inception-like Unit](https://mlsanity.com/paper/2103.13425/) (2021)
- [An Image is Worth 16x16 Words, What is a Video Worth?](https://mlsanity.com/paper/2103.13915/) (2021)
- [Fast Vision Transformers with HiLo Attention](https://mlsanity.com/paper/2205.13213/) (2022)
- [Wavelet Convolutions for Large Receptive Fields](https://mlsanity.com/paper/2407.05848/) (2024)
- [Towards Fast and Accurate Real-World Depth Super-Resolution: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2104.06174/) (2021)
- [Countering Malicious DeepFakes: Survey, Battleground, and Horizon](https://mlsanity.com/paper/2103.00218/) (2021)
- [AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter](https://mlsanity.com/paper/2104.12840/) (2021)
- [EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones](https://mlsanity.com/paper/2211.09703/) (2022)
- [Glance and Focus Networks for Dynamic Visual Recognition](https://mlsanity.com/paper/2201.03014/) (2022)
- [Compute-Efficient Deep Learning: Algorithmic Trends and Opportunities](https://mlsanity.com/paper/2210.06640/) (2022)
- [Frequency Perception Network for Camouflaged Object Detection](https://mlsanity.com/paper/2308.08924/) (2023)
- [Learning Frequency-aware Dynamic Network for Efficient Super-Resolution](https://mlsanity.com/paper/2103.08357/) (2021)
- [Learned Image Compression with Gaussian-Laplacian-Logistic Mixture Model and Concatenated Residual Modules](https://mlsanity.com/paper/2107.06463/) (2021)
- [Reversible Vision Transformers](https://mlsanity.com/paper/2302.04869/) (2023)
- [Dynamic Resolution Network](https://mlsanity.com/paper/2106.02898/) (2021)
- [TCGL: Temporal Contrastive Graph for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2112.03587/) (2021)
- [Frequency Decoupling for Motion Magnification via Multi-Level Isomorphic Architecture](https://mlsanity.com/paper/2403.07347/) (2024)
- [Unsupervised Anomaly Detection in Medical Images Using Masked Diffusion Model](https://mlsanity.com/paper/2305.19867/) (2023)
- [Dynamic Grained Encoder for Vision Transformers](https://mlsanity.com/paper/2301.03831/) (2023)
- [How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing](https://mlsanity.com/paper/2405.12474/) (2024)
- [Out-of-distribution Detection via Frequency-regularized Generative Models](https://mlsanity.com/paper/2208.09083/) (2022)

## Abstract (arXiv)

In natural images, information is conveyed at different frequencies where higher frequencies are usually encoded with fine details and lower frequencies are usually encoded with global structures. Similarly, the output feature maps of a convolution layer can also be seen as a mixture of information at different frequencies. In this work, we propose to factorize the mixed feature maps by their frequencies, and design a novel Octave Convolution (OctConv) operation to store and process feature maps that vary spatially "slower" at a lower spatial resolution reducing both memory and computation cost. Unlike existing multi-scale methods, OctConv is formulated as a single, generic, plug-and-play convolutional unit that can be used as a direct replacement of (vanilla) convolutions without any adjustments in the network architecture. It is also orthogonal and complementary to methods that suggest better topologies or reduce channel-wise redundancy like group or depth-wise convolutions. We experimentally show that by simply replacing convolutions with OctConv, we can consistently boost accuracy for both image and video recognition tasks, while reducing memory and computational cost. An OctConv-equipped ResNet-152 can achieve 82.9% top-1 classification accuracy on ImageNet with merely 22.2 GFLOPs.