# Deep Roots: Improving CNN Efficiency with Hierarchical Filter Groups

arXiv: https://arxiv.org/abs/1605.06489 · HTML: https://mlsanity.com/paper/1605.06489/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 May 2016.

## Built on by (most cited first, 30 of 32)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection](https://mlsanity.com/paper/2103.09096/) (2021)
- [LEAP: Learning Articulated Occupancy of People](https://mlsanity.com/paper/2104.06849/) (2021)
- [GET: Group Event Transformer for Event-Based Vision](https://mlsanity.com/paper/2310.02642/) (2023)
- [Scaling Forward Gradient With Local Losses](https://mlsanity.com/paper/2210.03310/) (2022)
- [Compressing Neural Networks: Towards Determining the Optimal Layer-wise Decomposition](https://mlsanity.com/paper/2107.11442/) (2021)
- [Applications and Techniques for Fast Machine Learning in Science](https://mlsanity.com/paper/2110.13041/) (2021)
- [Extremely Lightweight Quantization Robust Real-Time Single-Image Super Resolution for Mobile Devices](https://mlsanity.com/paper/2105.10288/) (2021)
- [HANet: A Hierarchical Attention Network for Change Detection With Bitemporal Very-High-Resolution Remote Sensing Images](https://mlsanity.com/paper/2404.09178/) (2024)
- [Dual-side Sparse Tensor Core](https://mlsanity.com/paper/2105.09564/) (2021)
- [Structured Sparsity Inducing Adaptive Optimizers for Deep Learning](https://mlsanity.com/paper/2102.03869/) (2021)
- [Neural Architecture Search as Program Transformation Exploration](https://mlsanity.com/paper/2102.06599/) (2021)
- [Making EfficientNet More Efficient: Exploring Batch-Independent Normalization, Group Convolutions and Reduced Resolution Training](https://mlsanity.com/paper/2106.03640/) (2021)
- [Ad2Attack: Adaptive Adversarial Attack on Real-Time UAV Tracking](https://mlsanity.com/paper/2203.01516/) (2022)
- [SYENet: A Simple Yet Effective Network for Multiple Low-Level Vision Tasks with Real-time Performance on Mobile Device](https://mlsanity.com/paper/2308.08137/) (2023)
- [Two-level Group Convolution](https://mlsanity.com/paper/2110.05060/) (2021)
- [DualConv: Dual Convolutional Kernels for Lightweight Deep Neural Networks](https://mlsanity.com/paper/2202.07481/) (2022)
- [Minimalistic Video Saliency Prediction via Efficient Decoder & Spatio Temporal Action Cues](https://mlsanity.com/paper/2502.00397/) (2025)
- [XCAT -- Lightweight Quantized Single Image Super-Resolution using Heterogeneous Group Convolutions and Cross Concatenation](https://mlsanity.com/paper/2208.14655/) (2022)
- [Boosting Convolutional Neural Networks with Middle Spectrum Grouped Convolution](https://mlsanity.com/paper/2304.06305/) (2023)
- [GroupBERT: Enhanced Transformer Architecture with Efficient Grouped Structures](https://mlsanity.com/paper/2106.05822/) (2021)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [Selecting Related Knowledge via Efficient Channel Attention for Online Continual Learning](https://mlsanity.com/paper/2209.04212/) (2022)
- [Design Automation for Fast, Lightweight, and Effective Deep Learning Models: A Survey](https://mlsanity.com/paper/2208.10498/) (2022)
- [Evaluation of Convolution Primitives for Embedded Neural Networks on 32-bit Microcontrollers](https://mlsanity.com/paper/2303.10702/) (2023)
- [Count, Decode and Fetch: A New Approach to Handwritten Chinese Character Error Correction](https://mlsanity.com/paper/2307.16253/) (2023)
- [My(o) Armband Leaks Passwords: An EMG and IMU Based Keylogging Side-Channel Attack](https://mlsanity.com/paper/2112.02382/) (2021)
- [LiPar: A Lightweight Parallel Learning Model for Practical In-Vehicle Network Intrusion Detection](https://mlsanity.com/paper/2311.08000/) (2023)
- [Cell Variational Information Bottleneck Network](https://mlsanity.com/paper/2403.15082/) (2024)
- [EditIQ: Automated Cinematic Editing of Static Wide-Angle Videos via Dialogue Interpretation and Saliency Cues](https://mlsanity.com/paper/2502.02172/) (2025)

## Abstract (arXiv)

We propose a new method for creating computationally efficient and compact convolutional neural networks (CNNs) using a novel sparse connection structure that resembles a tree root. This allows a significant reduction in computational cost and number of parameters compared to state-of-the-art deep CNNs, without compromising accuracy, by exploiting the sparsity of inter-layer filter dependencies. We validate our approach by using it to train more efficient variants of state-of-the-art CNN architectures, evaluated on the CIFAR10 and ILSVRC datasets. Our results show similar or higher accuracy than the baseline architectures with much less computation, as measured by CPU and GPU timings. For example, for ResNet 50, our model has 40% fewer parameters, 45% fewer floating point operations, and is 31% (12%) faster on a CPU (GPU). For the deeper ResNet 200 our model has 25% fewer floating point operations and 44% fewer parameters, while maintaining state-of-the-art accuracy. For GoogLeNet, our model has 7% fewer parameters and is 21% (16%) faster on a CPU (GPU).