mlsanityData to 7 Oct 2026

Multi-level Residual Networks from Dynamical Systems View

arXiv 1710.10348 ↗First posted 27 Oct 2017stat.ML cs.CVcited by 59

Not on the map. Why?

Abstract (arXiv)

Deep residual networks (ResNets) and their variants are widely used in many computer vision applications and natural language processing tasks. However, the theoretical principles for designing and training ResNets are still not fully understood. Recently, several points of view have emerged to try to interpret ResNet theoretically, such as unraveled view, unrolled iterative estimation and dynamical systems view. In this paper, we adopt the dynamical systems point of view, and analyze the lesioning properties of ResNet both theoretically and experimentally. Based on these analyses, we additionally propose a novel method for accelerating ResNet training. We apply the proposed method to train ResNets and Wide ResNets for three image classification benchmarks, reducing training time by more than 40% with superior or on-par accuracy.

Key facts. Cited by 59 papers in the map (data through 7 Oct 2026). First posted on arXiv 27 Oct 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 7; 2023: 10; 2024: 9; 2025: 10; 2026: 4.

59 papers.

Sort:

By area:

  1. 2023Learning to Grow Pretrained Models for Efficient Transformer Trainingcited by 81
  2. 2021Stabilizing Equilibrium Models by Jacobian Regularizationcited by 71
  3. 2021Encoding physics to learn reaction-diffusion processescited by 61
  4. 20251000 Layer Networks for Self-Supervised RL: Scaling Depth Can Enable New Goal-Reaching Capabilitiescited by 39
  5. 2022Automated Progressive Learning for Efficient Training of Vision Transformerscited by 36
  6. 2021Redesigning the Transformer Architecture with Insights from Multi-particle Dynamical Systemscited by 29
  7. 2021Constructing Neural Network-Based Models for Simulating Dynamical Systemscited by 29
  8. 2021Neural Delay Differential Equationscited by 26
  9. 2021The Elastic Lottery Ticket Hypothesiscited by 23
  10. 2023LEMON: Lossless model expansioncited by 23
  11. 2021Learning optimal multigrid smoothers via neural networkscited by 17
  12. 2022Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rulescited by 15
38 more
  1. 2021Stateful ODE-Nets using Basis Function Expansionscited by 14
  2. 2021Globally Convergent Multilevel Training of Deep Residual Networkscited by 11
  3. 2026ROCKET: Residual-Oriented Multi-Layer Alignment for Spatially-Aware Vision-Language-Action Modelscited by 11
  4. 2024Preparing Lessons for Progressive Training on Language Modelscited by 10
  5. 2025Neural equilibria for long-term prediction of nonlinear conservation lawscited by 10
  6. 2023Understanding Self-attention Mechanism via Dynamical System Perspectivecited by 9
  7. 2021Hard Encoding of Physics for Learning Spatiotemporal Dynamicscited by 8
  8. 2022A Recursively Recurrent Neural Network (R2N2) Architecture for Learning Iterative Algorithmscited by 8
  9. 2024Interplay between depth and width for interpolation in neural ODEscited by 8
  10. 2021Speeding up Deep Model Training by Sharing Weights and Then Unsharingcited by 7
  11. 2023NeFL: Nested Model Scaling for Federated Learning with System Heterogeneous Clientscited by 7
  12. 2021An Interpretive Constrained Linear Model for ResNet and MgNetcited by 6
  13. 2023Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Trainingcited by 6
  14. 2025HD-NDEs: Neural Differential Equations for Hallucination Detection in LLMscited by 6
  15. 2025LESA: Learnable LLM Layer Scaling-Upcited by 5
  16. 2021Multiscale Invertible Generative Networks for High-Dimensional Bayesian Inferencecited by 4
  17. 2022Neural Piecewise-Constant Delay Differential Equationscited by 4
  18. 2023AttNS: Attention-Inspired Numerical Solving For Limited Data Scenarioscited by 4
  19. 2024Efficient Training of Large Vision Models via Advanced Automated Progressive Learningcited by 4
  20. 2025Optimal Transport Depth Up-Scalingcited by 4
  21. 2021m-RevNet: Deep Reversible Neural Networks with Momentumcited by 3
  22. 2023Hierarchical deep learning-based adaptive time-stepping scheme for multiscale simulationscited by 3
  23. 2024When To Grow? A Fitting Risk-Aware Policy for Layer Growing in Deep Neural Networkscited by 3
  24. 2024Rethinking the Relationship between Recurrent and Non-Recurrent Neural Networks: A Study in Sparsitycited by 3
  25. 2025Recursive Bound-Constrained AdaGrad with Applications to Multilevel and Domain Decomposition Minimizationcited by 3
  26. 2021Training of deep residual networks with stochastic MG/OPTcited by 2
  27. 2024Representing Neural Network Layers as Linear Operations via Koopman Operator Theorycited by 2
  28. 2025STEP: Staged Parameter-Efficient Pre-training for Large Language Modelscited by 2
  29. 2021Meta-Solver for Neural Ordinary Differential Equationscited by 1
  30. 2021AFINet: Attentive Feature Integration Networks for Image Classificationcited by 1
  31. 2021Solving Image PDEs with a Shallow Networkcited by 1
  32. 2022Euler State Networks: Non-dissipative Reservoir Computingcited by 1
  33. 2023Neural Stochastic Differential Equations for Robust and Explainable Analysis of Electromagnetic Unintended Radiated Emissionscited by 1
  34. 2023SensLI: Sensitivity-Based Layer Insertion for Neural Networkscited by 1
  35. 2025PMNO: A novel physics guided multi-step neural operator predictor for partial differential equationscited by 1
  36. 2025Scaling depth capacity via zero/one-layer model expansioncited by 1
  37. 2026Multilevel Training for Kolmogorov Arnold Networkscited by 1
  38. 2021Residual networks classify inputs based on their neural transient dynamicscited by 0

50 most cited of 59 All 59 (JSON)