# IGCV3: Interleaved Low-Rank Group Convolutions for Efficient Deep Neural Networks

arXiv: https://arxiv.org/abs/1806.00178 · HTML: https://mlsanity.com/paper/1806.00178/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2018.

## Built on by (most cited first, 14 of 14)

- [Lite-HRNet: A Lightweight High-Resolution Network](https://mlsanity.com/paper/2104.06403/) (2021)
- [On the Connection between Local Attention and Dynamic Depth-wise Convolution](https://mlsanity.com/paper/2106.04263/) (2021)
- [MSRF-Net: A Multi-Scale Residual Fusion Network for Biomedical Image Segmentation](https://mlsanity.com/paper/2105.07451/) (2021)
- [CondenseNet V2: Sparse Feature Reactivation for Deep Networks](https://mlsanity.com/paper/2104.04382/) (2021)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [GMSRF-Net: An improved generalizability with global multi-scale residual fusion network for polyp segmentation](https://mlsanity.com/paper/2111.10614/) (2021)
- [Update Compression for Deep Neural Networks on the Edge](https://mlsanity.com/paper/2203.04516/) (2022)
- [FalconNet: Factorization for the Light-weight ConvNets](https://mlsanity.com/paper/2306.06365/) (2023)
- [Progressive Channel-Shrinking Network](https://mlsanity.com/paper/2304.00280/) (2023)
- [Learnable Heterogeneous Convolution: Learning both topology and strength](https://mlsanity.com/paper/2301.05440/) (2023)
- [Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution](https://mlsanity.com/paper/2304.06305/) (2023)
- [Pre-Trained Model Recommendation for Downstream Fine-tuning](https://mlsanity.com/paper/2403.06382/) (2024)
- [Shift-and-Balance Attention](https://mlsanity.com/paper/2103.13080/) (2021)
- [Learning from Pattern Completion: Self-supervised Controllable Generation](https://mlsanity.com/paper/2409.18694/) (2024)

## Abstract (arXiv)

In this paper, we are interested in building lightweight and efficient convolutional neural networks. Inspired by the success of two design patterns, composition of structured sparse kernels, e.g., interleaved group convolutions (IGC), and composition of low-rank kernels, e.g., bottle-neck modules, we study the combination of such two design patterns, using the composition of structured sparse low-rank kernels, to form a convolutional kernel. Rather than introducing a complementary condition over channels, we introduce a loose complementary condition, which is formulated by imposing the complementary condition over super-channels, to guide the design for generating a dense convolutional kernel. The resulting network is called IGCV3. We empirically demonstrate that the combination of low-rank and sparse kernels boosts the performance and the superiority of our proposed approach to the state-of-the-arts, IGCV2 and MobileNetV2 over image classification on CIFAR and ImageNet and object detection on COCO.