# Decoupled Dynamic Filter Networks

arXiv: https://arxiv.org/abs/2104.14107 · HTML: https://mlsanity.com/paper/2104.14107/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 60 references that are in the map.
- First posted on arXiv 29 Apr 2021.

## Builds on (most cited first, 30 of 40)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [MnasNet: Platform-Aware Neural Architecture Search for Mobile](https://mlsanity.com/paper/1807.11626/) (2018)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Residual Attention Network for Image Classification](https://mlsanity.com/paper/1704.06904/) (2017)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)
- [Attention Augmented Convolutional Networks](https://mlsanity.com/paper/1904.09925/) (2019)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [Dynamic Filter Networks](https://mlsanity.com/paper/1605.09673/) (2016)
- [Conditional Convolutions for Instance Segmentation](https://mlsanity.com/paper/2003.05664/) (2020)
- [CondConv: Conditionally Parameterized Convolutions for Efficient Inference](https://mlsanity.com/paper/1904.04971/) (2019)
- [BAM: Bottleneck Attention Module](https://mlsanity.com/paper/1807.06514/) (2018)
- [Pixel-Adaptive Convolutional Neural Networks](https://mlsanity.com/paper/1904.05373/) (2019)
- [CARAFE: Content-Aware ReAssembly of FEatures](https://mlsanity.com/paper/1905.02188/) (2019)

## Built on by (most cited first, 30 of 53)

- [Delivering Arbitrary-Modal Semantic Segmentation](https://mlsanity.com/paper/2303.01480/) (2023)
- [Dynamic Spatial Propagation Network for Depth Completion](https://mlsanity.com/paper/2202.09769/) (2022)
- [TAda! Temporally-Adaptive Convolutions for Video Understanding](https://mlsanity.com/paper/2110.06178/) (2021)
- [DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalization](https://mlsanity.com/paper/2209.05227/) (2022)
- [Frequency-Adaptive Dilated Convolution for Semantic Segmentation](https://mlsanity.com/paper/2403.05369/) (2024)
- [Domain and Content Adaptive Convolution based Multi-Source Domain Generalization for Medical Image Segmentation](https://mlsanity.com/paper/2109.05676/) (2021)
- [PointConvFormer: Revenge of the Point-based Convolution](https://mlsanity.com/paper/2208.02879/) (2022)
- [Scaled ReLU Matters for Training Vision Transformers](https://mlsanity.com/paper/2109.03810/) (2021)
- [Adaptive Dynamic Filtering Network for Image Denoising](https://mlsanity.com/paper/2211.12051/) (2022)
- [Centering the Value of Every Modality: Towards Efficient and Resilient Modality-agnostic Semantic Segmentation](https://mlsanity.com/paper/2407.11344/) (2024)
- [Towards Real-world Event-guided Low-light Video Enhancement and Deblurring](https://mlsanity.com/paper/2408.14916/) (2024)
- [Content-Adaptive Non-Local Convolution for Remote Sensing Pansharpening](https://mlsanity.com/paper/2404.07543/) (2024)
- [Event-Aware Multimodal Mobility Nowcasting](https://mlsanity.com/paper/2112.08443/) (2021)
- [ELSA: Enhanced Local Self-Attention for Vision Transformer](https://mlsanity.com/paper/2112.12786/) (2021)
- [Frequency Disentangled Features in Neural Image Compression](https://mlsanity.com/paper/2308.02620/) (2023)
- [U3M: Unbiased Multiscale Modal Fusion Model for Multimodal Semantic Segmentation](https://mlsanity.com/paper/2405.15365/) (2024)
- [Fast Object Placement Assessment](https://mlsanity.com/paper/2205.14280/) (2022)
- [MMSFormer: Multimodal Transformer for Material and Semantic Segmentation](https://mlsanity.com/paper/2309.04001/) (2023)
- [Decomposed Guided Dynamic Filters for Efficient RGB-Guided Depth Completion](https://mlsanity.com/paper/2309.02043/) (2023)
- [Adaptive Rectangular Convolution for Remote Sensing Pansharpening](https://mlsanity.com/paper/2503.00467/) (2025)
- [CMTA: Cross-Modal Temporal Alignment for Event-guided Video Deblurring](https://mlsanity.com/paper/2408.14930/) (2024)
- [Condition-Adaptive Graph Convolution Learning for Skeleton-Based Gait Recognition](https://mlsanity.com/paper/2308.06707/) (2023)
- [TVConv: Efficient Translation Variant Convolution for Layout-aware Visual Processing](https://mlsanity.com/paper/2203.10489/) (2022)
- [Temporally-Adaptive Models for Efficient Video Understanding](https://mlsanity.com/paper/2308.05787/) (2023)
- [ShareCMP: Polarization-Aware RGB-P Semantic Segmentation](https://mlsanity.com/paper/2312.03430/) (2023)
- [DESTINE: Dynamic Goal Queries with Temporal Transductive Alignment for Trajectory Prediction](https://mlsanity.com/paper/2310.07438/) (2023)
- [Scene Prior Filtering for Depth Super-Resolution](https://mlsanity.com/paper/2402.13876/) (2024)
- [Complementary Advantages: Exploiting Cross-Field Frequency Correlation for NIR-Assisted Image Denoising](https://mlsanity.com/paper/2412.16645/) (2024)
- [TADFormer : Task-Adaptive Dynamic Transformer for Efficient Multi-Task Learning](https://mlsanity.com/paper/2501.04293/) (2025)
- [Learning Instance and Task-Aware Dynamic Kernels for Few Shot Learning](https://mlsanity.com/paper/2112.03494/) (2021)

## Abstract (arXiv)

Convolution is one of the basic building blocks of CNN architectures. Despite its common use, standard convolution has two main shortcomings: Content-agnostic and Computation-heavy. Dynamic filters are content-adaptive, while further increasing the computational overhead. Depth-wise convolution is a lightweight variant, but it usually leads to a drop in CNN performance or requires a larger number of channels. In this work, we propose the Decoupled Dynamic Filter (DDF) that can simultaneously tackle both of these shortcomings. Inspired by recent advances in attention, DDF decouples a depth-wise dynamic filter into spatial and channel dynamic filters. This decomposition considerably reduces the number of parameters and limits computational costs to the same level as depth-wise convolution. Meanwhile, we observe a significant boost in performance when replacing standard convolution with DDF in classification networks. ResNet50 / 101 get improved by 1.9% and 1.3% on the top-1 accuracy, while their computational costs are reduced by nearly half. Experiments on the detection and joint upsampling networks also demonstrate the superior performance of the DDF upsampling variant (DDF-Up) in comparison with standard convolution and specialized content-adaptive layers.