mlsanityData to 7 Oct 2026

Analyze and Design Network Architectures by Recursion Formulas

arXiv 2108.08689 ↗First posted 18 Aug 2021cs.LG cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

The effectiveness of shortcut/skip-connection has been widely verified, which inspires massive explorations on neural architecture design. This work attempts to find an effective way to design new network architectures. It is discovered that the main difference between network architectures can be reflected in their recursion formulas. Based on this, a methodology is proposed to design novel network architectures from the perspective of mathematical formulas. Afterwards, a case study is provided to generate an improved architecture based on ResNet. Furthermore, the new architecture is compared with ResNet and then tested on ResNet-based networks. Massive experiments are conducted on CIFAR and ImageNet, which witnesses the significant performance improvements provided by the architecture.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 19 of its 31 references that are in the map. First posted on arXiv 18 Aug 2021.

Builds on

19 of 31 references.

References · 19

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Densely Connected Convolutional Networkscited by 6,691
  3. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2017Squeeze-and-Excitation Networkscited by 4,763
  8. 2014Going Deeper with Convolutionscited by 4,231
  9. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  10. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  11. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  12. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
7 more
  1. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  2. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  3. 2015Highway Networkscited by 257
  4. 2020Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depthcited by 241
  5. 2016Wider or Deeper: Revisiting the ResNet Model for Visual Recognitioncited by 184
  6. 2021Revisiting ResNets: Improved Training and Scaling Strategiescited by 158
  7. 2016Feedback Networkscited by 29

Built on by

Not cited yet.