# Designing Network Design Strategies Through Gradient Path Analysis

arXiv: https://arxiv.org/abs/2211.04800 · HTML: https://mlsanity.com/paper/2211.04800/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 40 references that are in the map.
- First posted on arXiv 9 Nov 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs](https://mlsanity.com/paper/1606.00915/) (2016)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [Deep Networks with Stochastic Depth](https://mlsanity.com/paper/1603.09382/) (2016)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [Selective Kernel Networks](https://mlsanity.com/paper/1903.06586/) (2019)
- [CSPNet: A New Backbone that can Enhance Learning Capability of CNN](https://mlsanity.com/paper/1911.11929/) (2019)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [FractalNet: Ultra-Deep Neural Networks without Residuals](https://mlsanity.com/paper/1605.07648/) (2016)
- [Scaled-YOLOv4: Scaling Cross Stage Partial Network](https://mlsanity.com/paper/2011.08036/) (2020)
- [CenterMask : Real-Time Anchor-Free Instance Segmentation](https://mlsanity.com/paper/1911.06667/) (2019)
- [An Energy and GPU-Computation Efficient Backbone Network for Real-Time Object Detection](https://mlsanity.com/paper/1904.09730/) (2019)
- [Receptive Field Block Net for Accurate and Fast Object Detection](https://mlsanity.com/paper/1711.07767/) (2017)
- [Disentangled Non-Local Neural Networks](https://mlsanity.com/paper/2006.06668/) (2020)
- [You Only Learn One Representation: Unified Network for Multiple Tasks](https://mlsanity.com/paper/2105.04206/) (2021)

## Built on by (most cited first, 30 of 32)

- [YOLOv10: Real-Time End-to-End Object Detection](https://mlsanity.com/paper/2405.14458/) (2024)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [YOLOv12: Attention-Centric Real-Time Object Detectors](https://mlsanity.com/paper/2502.12524/) (2025)
- [A Comprehensive Review of YOLO Architectures in Computer Vision: From YOLOv1 to YOLOv8 and YOLO-NAS](https://mlsanity.com/paper/2304.00501/) (2023)
- [DRCT: Saving Image Super-resolution away from Information Bottleneck](https://mlsanity.com/paper/2404.00722/) (2024)
- [YOLO-MS: Rethinking Multi-Scale Representation Learning for Real-time Object Detection](https://mlsanity.com/paper/2308.05480/) (2023)
- [Hyper-YOLO: When Visual Object Detection Meets Hypergraph Computation](https://mlsanity.com/paper/2408.04804/) (2024)
- [YOLOv9 for Fracture Detection in Pediatric Wrist Trauma X-ray Images](https://mlsanity.com/paper/2403.11249/) (2024)
- [YOLOv8-AM: YOLOv8 Based on Effective Attention Mechanisms for Pediatric Wrist Fracture Detection](https://mlsanity.com/paper/2402.09329/) (2024)
- [YOLOv1 to YOLOv10: The fastest and most accurate real-time object detection systems](https://mlsanity.com/paper/2408.09332/) (2024)
- [RCS-YOLO: A Fast and High-Accuracy Object Detector for Brain Tumor Detection](https://mlsanity.com/paper/2307.16412/) (2023)
- [YOLOR-Based Multi-Task Learning](https://mlsanity.com/paper/2309.16921/) (2023)
- [SOAR: Advancements in Small Body Object Detection for Aerial Imagery Using State Space Models and Programmable Gradients](https://mlsanity.com/paper/2405.01699/) (2024)
- [YOLOv8-ResCBAM: YOLOv8 Based on An Effective Attention Module for Pediatric Wrist Fracture Detection](https://mlsanity.com/paper/2409.18826/) (2024)
- [YOLOCS: Object Detection based on Dense Channel Compression for Feature Spatial Solidification](https://mlsanity.com/paper/2305.04170/) (2023)
- [Iterative Filter Pruning for Concatenation-based CNN Architectures](https://mlsanity.com/paper/2405.03715/) (2024)
- [What is YOLOv9: An In-Depth Exploration of the Internal Features of the Next-Generation Object Detector](https://mlsanity.com/paper/2409.07813/) (2024)
- [A Decade of You Only Look Once (YOLO) for Object Detection: A Review](https://mlsanity.com/paper/2504.18586/) (2025)
- [FedPylot: Navigating Federated Learning for Real-Time Object Detection in Internet of Vehicles](https://mlsanity.com/paper/2406.03611/) (2024)
- [Finetuning YOLOv9 for Vehicle Detection: Deep Learning for Intelligent Transportation Systems in Dhaka, Bangladesh](https://mlsanity.com/paper/2410.08230/) (2024)
- [Performance Optimization of YOLO-FEDER FusionNet for Robust Drone Detection in Visually Complex Environments](https://mlsanity.com/paper/2509.14012/) (2025)
- [Underwater target detection based on improved YOLOv7](https://mlsanity.com/paper/2302.06939/) (2023)
- [CST-YOLO: A Novel Method for Blood Cell Detection Based on Improved YOLOv7 and CNN-Swin Transformer](https://mlsanity.com/paper/2306.14590/) (2023)
- [PK-YOLO: Pretrained Knowledge Guided YOLO for Brain Tumor Detection in Multiplanar MRI Slices](https://mlsanity.com/paper/2410.21822/) (2024)
- [Real-time Video Prediction With Fast Video Interpolation Model and Prediction Training](https://mlsanity.com/paper/2503.23185/) (2025)
- [Q-YOLOP: Quantization-aware You Only Look Once for Panoptic Driving Perception](https://mlsanity.com/paper/2307.04537/) (2023)
- [RepVGG-GELAN: Enhanced GELAN with VGG-STYLE ConvNets for Brain Tumour Detection](https://mlsanity.com/paper/2405.03541/) (2024)
- [YOLO-ELA: Efficient Local Attention Modeling for High-Performance Real-Time Insulator Defect Detection](https://mlsanity.com/paper/2410.11727/) (2024)
- [YOLO-PRO: Enhancing Instance-Specific Object Detection with Full-Channel Global Self-Attention](https://mlsanity.com/paper/2503.02348/) (2025)
- [Two-Stage Framework for Efficient UAV-Based Wildfire Video Analysis with Adaptive Compression and Fire Source Detection](https://mlsanity.com/paper/2508.16739/) (2025)

## Abstract (arXiv)

Designing a high-efficiency and high-quality expressive network architecture has always been the most important research topic in the field of deep learning. Most of today's network design strategies focus on how to integrate features extracted from different layers, and how to design computing units to effectively extract these features, thereby enhancing the expressiveness of the network. This paper proposes a new network design strategy, i.e., to design the network architecture based on gradient path analysis. On the whole, most of today's mainstream network design strategies are based on feed forward path, that is, the network architecture is designed based on the data path. In this paper, we hope to enhance the expressive ability of the trained model by improving the network learning ability. Due to the mechanism driving the network parameter learning is the backward propagation algorithm, we design network design strategies based on back propagation path. We propose the gradient path design strategies for the layer-level, the stage-level, and the network-level, and the design strategies are proved to be superior and feasible from theoretical analysis and experiments.