# DO-Conv: Depthwise Over-parameterized Convolutional Layer

arXiv: https://arxiv.org/abs/2006.12030 · HTML: https://mlsanity.com/paper/2006.12030/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2020.

## Built on by (most cited first, 28 of 28)

- [RepVGG: Making VGG-style ConvNets Great Again](https://mlsanity.com/paper/2101.03697/) (2021)
- [U-KAN Makes Strong Backbone for Medical Image Segmentation and Generation](https://mlsanity.com/paper/2406.02918/) (2024)
- [Diverse Branch Block: Building a Convolution as an Inception-like Unit](https://mlsanity.com/paper/2103.13425/) (2021)
- [What Makes Convolutional Models Great on Long Sequence Modeling?](https://mlsanity.com/paper/2210.09298/) (2022)
- [Initialization and Regularization of Factorized Neural Layers](https://mlsanity.com/paper/2105.01029/) (2021)
- [On the Convergence of Shallow Neural Network Training with Randomly Masked Neurons](https://mlsanity.com/paper/2112.02668/) (2021)
- [RefConv: Re-parameterized Refocusing Convolution for Powerful ConvNets](https://mlsanity.com/paper/2310.10563/) (2023)
- [NTIRE 2025 Challenge on RAW Image Restoration and Super-Resolution](https://mlsanity.com/paper/2506.02197/) (2025)
- [Sparse-IFT: Sparse Iso-FLOP Transformations for Maximizing Training Efficiency](https://mlsanity.com/paper/2303.11525/) (2023)
- [Restructurable Activation Networks](https://mlsanity.com/paper/2208.08562/) (2022)
- [Online Convolutional Re-parameterization](https://mlsanity.com/paper/2204.00826/) (2022)
- [Boosting Pruned Networks with Linear Over-parameterization](https://mlsanity.com/paper/2204.11444/) (2022)
- [DAPONet: A Dual Attention and Partially Overparameterized Network for Real-Time Road Damage Detection](https://mlsanity.com/paper/2409.01604/) (2024)
- [ASR: Attention-alike Structural Re-parameterization](https://mlsanity.com/paper/2304.06345/) (2023)
- [Conv-INR: Convolutional Implicit Neural Representation for Multimodal Visual Signals](https://mlsanity.com/paper/2406.04249/) (2024)
- [Parameter Expanded Stochastic Gradient Markov Chain Monte Carlo](https://mlsanity.com/paper/2503.00699/) (2025)
- [Novel Extraction of Discriminative Fine-Grained Feature to Improve Retinal Vessel Segmentation](https://mlsanity.com/paper/2505.03896/) (2025)
- [Lecture Notes on Linear Neural Networks: A Tale of Optimization and Generalization in Deep Learning](https://mlsanity.com/paper/2408.13767/) (2024)
- [Shallow Network Based on Depthwise Over-Parameterized Convolution for Hyperspectral Image Classification](https://mlsanity.com/paper/2112.00250/) (2021)
- [Utilizing Excess Resources in Training Neural Networks](https://mlsanity.com/paper/2207.05532/) (2022)
- [High-Performance Inference Graph Convolutional Networks for Skeleton-Based Action Recognition](https://mlsanity.com/paper/2305.18710/) (2023)
- [Learning Compact Neural Networks with Deep Overparameterised Multitask Learning](https://mlsanity.com/paper/2308.13300/) (2023)
- [Hybrid Pooling and Convolutional Network for Improving Accuracy and Training Convergence Speed in Object Detection](https://mlsanity.com/paper/2401.01134/) (2024)
- [MFDS-Net: Multi-Scale Feature Depth-Supervised Network for Remote Sensing Change Detection with Global Semantic and Detail Information](https://mlsanity.com/paper/2405.01065/) (2024)
- [Gap Preserving Distillation by Building Bidirectional Mappings with A Dynamic Teacher](https://mlsanity.com/paper/2410.04140/) (2024)
- [Semi-KAN: KAN Provides an Effective Representation for Semi-Supervised Learning in Medical Image Segmentation](https://mlsanity.com/paper/2503.14983/) (2025)
- [UKANFormer: Noise-Robust Semantic Segmentation for Coral Reef Mapping via a Kolmogorov-Arnold Network-Transformer Hybrid](https://mlsanity.com/paper/2510.16730/) (2025)
- [DB-KAUNet: An Adaptive Dual Branch Kolmogorov-Arnold UNet for Retinal Vessel Segmentation](https://mlsanity.com/paper/2512.01657/) (2025)

## Abstract (arXiv)

Convolutional layers are the core building blocks of Convolutional Neural Networks (CNNs). In this paper, we propose to augment a convolutional layer with an additional depthwise convolution, where each input channel is convolved with a different 2D kernel. The composition of the two convolutions constitutes an over-parameterization, since it adds learnable parameters, while the resulting linear operation can be expressed by a single convolution layer. We refer to this depthwise over-parameterized convolutional layer as DO-Conv. We show with extensive experiments that the mere replacement of conventional convolutional layers with DO-Conv layers boosts the performance of CNNs on many classical vision tasks, such as image classification, detection, and segmentation. Moreover, in the inference phase, the depthwise convolution is folded into the conventional convolution, reducing the computation to be exactly equivalent to that of a convolutional layer without over-parameterization. As DO-Conv introduces performance gains without incurring any computational complexity increase for inference, we advocate it as an alternative to the conventional convolutional layer. We open-source a reference implementation of DO-Conv in Tensorflow, PyTorch and GluonCV at https://github.com/yangyanli/DO-Conv.