# MixConv: Mixed Depthwise Convolutional Kernels

arXiv: https://arxiv.org/abs/1907.09595 · HTML: https://mlsanity.com/paper/1907.09595/

## Key facts

- Cited by 108 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jul 2019.

## Built on by (most cited first, 30 of 108)

- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [MobileViT: Light-weight, General-purpose, and Mobile-friendly Vision Transformer](https://mlsanity.com/paper/2110.02178/) (2021)
- [DeepViT: Towards Deeper Vision Transformer](https://mlsanity.com/paper/2103.11886/) (2021)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)
- [Mobile-Former: Bridging MobileNet and Transformer](https://mlsanity.com/paper/2108.05895/) (2021)
- [Zero-Cost Proxies for Lightweight NAS](https://mlsanity.com/paper/2101.08134/) (2021)
- [EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention](https://mlsanity.com/paper/2305.07027/) (2023)
- [EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba](https://mlsanity.com/paper/2403.09977/) (2024)
- [CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention](https://mlsanity.com/paper/2108.00154/) (2021)
- [RepViT: Revisiting Mobile CNN From ViT Perspective](https://mlsanity.com/paper/2307.09283/) (2023)
- [EdgeViTs: Competing Light-weight CNNs on Mobile Devices with Vision Transformers](https://mlsanity.com/paper/2205.03436/) (2022)
- [MobileOne: An Improved One millisecond Mobile Backbone](https://mlsanity.com/paper/2206.04040/) (2022)
- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [InceptionNeXt: When Inception Meets ConvNeXt](https://mlsanity.com/paper/2303.16900/) (2023)
- [Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition](https://mlsanity.com/paper/2211.11943/) (2022)
- [Can GPT-4 Perform Neural Architecture Search?](https://mlsanity.com/paper/2304.10970/) (2023)
- [Quantum machine learning for image classification](https://mlsanity.com/paper/2304.09224/) (2023)
- [The Lie Derivative for Measuring Learned Equivariance](https://mlsanity.com/paper/2210.02984/) (2022)
- [FiLo: Zero-Shot Anomaly Detection by Fine-Grained Description and High-Quality Localization](https://mlsanity.com/paper/2404.13671/) (2024)
- [Scale-Aware Modulation Meet Transformer](https://mlsanity.com/paper/2307.08579/) (2023)
- [HR-NAS: Searching Efficient High-Resolution Neural Architectures with Lightweight Transformers](https://mlsanity.com/paper/2106.06560/) (2021)
- [PP-OCRv2: Bag of Tricks for Ultra Lightweight OCR System](https://mlsanity.com/paper/2109.03144/) (2021)
- [Lightweight Deep Learning for Resource-Constrained Environments: A Survey](https://mlsanity.com/paper/2404.07236/) (2024)
- [MixFaceNets: Extremely Efficient Face Recognition Networks](https://mlsanity.com/paper/2107.13046/) (2021)
- [PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation](https://mlsanity.com/paper/2108.10710/) (2021)
- [Revisiting Dynamic Convolution via Matrix Decomposition](https://mlsanity.com/paper/2103.08756/) (2021)
- [Pinwheel-shaped Convolution and Scale-based Dynamic Loss for Infrared Small Target Detection](https://mlsanity.com/paper/2412.16986/) (2024)
- [EdgeFace: Efficient Face Recognition Model for Edge Devices](https://mlsanity.com/paper/2307.01838/) (2023)
- [Beyond Fixation: Dynamic Window Visual Transformer](https://mlsanity.com/paper/2203.12856/) (2022)
- [Revisiting Residual Networks for Adversarial Robustness: An Architectural Perspective](https://mlsanity.com/paper/2212.11005/) (2022)

## Abstract (arXiv)

Depthwise convolution is becoming increasingly popular in modern efficient ConvNets, but its kernel size is often overlooked. In this paper, we systematically study the impact of different kernel sizes, and observe that combining the benefits of multiple kernel sizes can lead to better accuracy and efficiency. Based on this observation, we propose a new mixed depthwise convolution (MixConv), which naturally mixes up multiple kernel sizes in a single convolution. As a simple drop-in replacement of vanilla depthwise convolution, our MixConv improves the accuracy and efficiency for existing MobileNets on both ImageNet classification and COCO object detection. To demonstrate the effectiveness of MixConv, we integrate it into AutoML search space and develop a new family of models, named as MixNets, which outperform previous mobile models including MobileNetV2 [20] (ImageNet top-1 accuracy +4.2%), ShuffleNetV2 [16] (+3.5%), MnasNet [26] (+1.3%), ProxylessNAS [2] (+2.2%), and FBNet [27] (+2.0%). In particular, our MixNet-L achieves a new state-of-the-art 78.9% ImageNet top-1 accuracy under typical mobile settings (<600M FLOPS). Code is at https://github.com/ tensorflow/tpu/tree/master/models/official/mnasnet/mixnet