mlsanityData to 7 Oct 2026

Dynamic Convolution: Attention over Convolution Kernels

arXiv 1912.03458 ↗First posted 7 Dec 2019cs.CVcited by 355

Not on the map. Why?

Abstract (arXiv)

Light-weight convolutional neural networks (CNNs) suffer performance degradation as their low computational budgets constrain both the depth (number of convolution layers) and the width (number of channels) of CNNs, resulting in limited representation capability. To address this issue, we present Dynamic Convolution, a new design that increases model complexity without increasing the network depth or width. Instead of using a single convolution kernel per layer, dynamic convolution aggregates multiple parallel convolution kernels dynamically based upon their attentions, which are input dependent. Assembling multiple kernels is not only computationally efficient due to the small kernel size, but also has more representation power since these kernels are aggregated in a non-linear way via attention. By simply using dynamic convolution for the state-of-the-art architecture MobileNetV3-Small, the top-1 accuracy of ImageNet classification is boosted by 2.9% with only 4% additional FLOPs and 2.9 AP gain is achieved on COCO keypoint detection.

Key facts. Cited by 355 papers in the map (data through 7 Oct 2026). First posted on arXiv 7 Dec 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 52; 2022: 73; 2023: 71; 2024: 61; 2025: 56; 2026: 42.

355 papers.

Sort:

By area:

  1. 2021Going deeper with Image Transformerscited by 759
  2. 2021LeViT: a Vision Transformer in ConvNet's Clothing for Faster Inferencecited by 475
  3. 2021Conditional DETR for Fast Training Convergencecited by 422
  4. 2021Dynamic Neural Networks: A Surveycited by 405
  5. 2021Channel-wise Topology Refinement Graph Convolution for Skeleton-Based Action Recognitioncited by 317
  6. 2021Mobile-Former: Bridging MobileNet and Transformercited by 234
  7. 2021Attention Mechanisms in Computer Vision: A Surveycited by 232
  8. 2023A Survey on Deep Neural Network Pruning-Taxonomy, Comparison, Analysis, and Recommendationscited by 200
  9. 2022Focal Modulation Networkscited by 186
  10. 2022Focal Sparse Convolutional Networks for 3D Object Detectioncited by 142
  11. 2022EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformerscited by 116
  12. 2022HorNet: Efficient High-Order Spatial Interactions with Recursive Gated Convolutionscited by 103
38 more
  1. 2023Large Selective Kernel Network for Remote Sensing Object Detectioncited by 98
  2. 2022Efficient and Degradation-Adaptive Network for Real-World Image Super-Resolutioncited by 96
  3. 2021Lite-HRNet: A Lightweight High-Resolution Networkcited by 91
  4. 2023End-to-End Vectorized HD-map Construction with Piecewise Bezier Curvecited by 80
  5. 2022Dynamic Spatial Propagation Network for Depth Completioncited by 70
  6. 2022Zero-Shot Temporal Action Detection via Vision-Language Promptingcited by 65
  7. 2022Sparse Instance Activation for Real-Time Instance Segmentationcited by 63
  8. 2023Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generationcited by 61
  9. 2022Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Versioncited by 59
  10. 2023UniHCP: A Unified Model for Human-Centric Perceptionscited by 59
  11. 2022Omni-Dimensional Dynamic Convolutioncited by 55
  12. 2024Rewrite the Starscited by 55
  13. 2023Towards Stability of Autoregressive Neural Operatorscited by 54
  14. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  15. 2021Decoupled Dynamic Filter Networkscited by 52
  16. 2023Representing Volumetric Videos as Dynamic MLP Mapscited by 51
  17. 2022Towards Robust Referring Video Object Segmentation with Cyclic Relational Consensuscited by 49
  18. 2021AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognitioncited by 48
  19. 2023A Survey on Few-Shot Class-Incremental Learningcited by 48
  20. 2021Deep Learning for HDR Imaging: State-of-the-Art and Future Trendscited by 45
  21. 2023Adaptive Frequency Filters As Efficient Global Token Mixerscited by 45
  22. 2024LSKNet: A Foundation Lightweight Backbone for Remote Sensingcited by 44
  23. 2022Dynamic Prototype Convolution Network for Few-Shot Semantic Segmentationcited by 43
  24. 2024UnO: Unsupervised Occupancy Fields for Perception and Forecastingcited by 40
  25. 2024Spatial-Mamba: Effective Visual State Space Models via Structure-aware State Fusioncited by 40
  26. 2021DRB-GAN: A Dynamic ResBlock Generative Adversarial Network for Artistic Style Transfercited by 38
  27. 2022Multi-stage image denoising with the wavelet transformcited by 37
  28. 2023HyperE2VID: Improving Event-Based Video Reconstruction via Hypernetworkscited by 37
  29. 2021TAda! Temporally-Adaptive Convolutions for Video Understandingcited by 36
  30. 2023Merging Experts into One: Improving Computational Efficiency of Mixture of Expertscited by 36
  31. 2021Revisiting Dynamic Convolution via Matrix Decompositioncited by 35
  32. 2022MISF: Multi-level Interactive Siamese Filtering for High-Fidelity Image Inpaintingcited by 35
  33. 2022DUET: A Tuning-Free Device-Cloud Collaborative Parameters Generation Framework for Efficient Device Model Generalizationcited by 35
  34. 2023Language Adaptive Weight Generation for Multi-task Visual Groundingcited by 35
  35. 2023DilateFormer: Multi-Scale Dilated Transformer for Visual Recognitioncited by 34
  36. 2022Deep Depth Completion from Extremely Sparse Data: A Surveycited by 33
  37. 2023KBNet: Kernel Basis Network for Image Restorationcited by 33
  38. 2022Designing Network Design Strategies Through Gradient Path Analysiscited by 32

50 most cited of 355 All 355 (JSON)