mlsanityData to 7 Oct 2026

An Interpretive Constrained Linear Model for ResNet and MgNet

arXiv 2112.07441 ↗First posted 14 Dec 2021cs.CV cs.AI cs.LGcited by 6

Not on the map. Why?

Abstract (arXiv)

We propose a constrained linear data-feature-mapping model as an interpretable mathematical model for image classification using a convolutional neural network (CNN). From this viewpoint, we establish detailed connections between the traditional iterative schemes for linear systems and the architectures of the basic blocks of ResNet- and MgNet-type models. Using these connections, we present some modified ResNet models that compared with the original models have fewer parameters and yet can produce more accurate results, thereby demonstrating the validity of this constrained learning data-feature-mapping assumption. Based on this assumption, we further propose a general data-feature iterative scheme to show the rationality of MgNet. We also provide a systematic numerical study on MgNet to show its success and advantages in image classification problems and demonstrate its advantages in comparison with established networks.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 35 of its 56 references that are in the map. First posted on arXiv 14 Dec 2021.

Builds on

35 of 56 references.

References · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  5. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  6. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  7. 2016Densely Connected Convolutional Networkscited by 6,691
  8. 2014Going Deeper with Convolutionscited by 4,231
  9. 2018Neural Ordinary Differential Equationscited by 3,733
  10. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  11. 2016Wide Residual Networkscited by 3,057
  12. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
23 more
  1. 2015Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Codingcited by 2,291
  2. 2016V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentationcited by 2,284
  3. 2016Identity Mappings in Deep Residual Networkscited by 1,960
  4. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  5. 2016FractalNet: Ultra-Deep Neural Networks without Residualscited by 216
  6. 2017The Reversible Residual Network: Backpropagation Without Storing Activationscited by 215
  7. 2017Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equationscited by 203
  8. 2019What Can ResNet Learn Efficiently, Going Beyond Kernels?cited by 91
  9. 2018ResNet with one-neuron hidden layers is a Universal Approximatorcited by 78
  10. 2019Learning Neural PDE Solvers with Convergence Guaranteescited by 77
  11. 2020Why Do Deep Residual Networks Generalize Better than Deep Feedforward Networks? -- A Neural Tangent Kernel Perspectivecited by 60
  12. 2017Multi-level Residual Networks from Dynamical Systems Viewcited by 59
  13. 2019MgNet: A Unified Framework of Multigrid and Convolutional Neural Networkcited by 41
  14. 2016PolyNet: A Pursuit of Structural Diversity in Very Deep Networkscited by 38
  15. 2017Learning Deep ResNet Blocks Sequentially using Boosting Theorycited by 32
  16. 2019SCAN: A Scalable Neural Networks Framework Towards Compact and Efficient Modelscited by 22
  17. 2020Kernel-Based Smoothness Analysis of Residual Networkscited by 19
  18. 2021Approximation Properties of Deep ReLU CNNscited by 15
  19. 2018Functional Gradient Boosting based on Residual Network Perceptioncited by 14
  20. 2018Optimization Algorithm Inspired Deep Neural Network Structure Designcited by 14
  21. 2017Learning across scales - A multiscale method for Convolution Neural Networkscited by 5
  22. 2018Make ℓ1 Regularization Effective in Training Sparse CNNcited by 4
  23. 2016The Loss Surface of Residual Networks: Ensembles and the Role of Batch Normalizationcited by 1

Built on by

Cited by year

Citing papers per year: 2022: 1; 2023: 3; 2024: 1; 2025: 1.

6 papers.

  1. 2023MgNO: Efficient Parameterization of Linear Operators via Multigridcited by 40
  2. 2024M2NO: An Efficient Multi-Resolution Operator Framework for Dynamic Multi-Scale PDE Solverscited by 6
  3. 2023An Enhanced V-cycle MgNet Model for Operator Learning in Numerical Partial Differential Equationscited by 4
  4. 2025Self-composing neural operators for high-frequency and multiscale PDE surrogatescited by 3
  5. 2022MGiaD: Multigrid in all dimensions. Efficiency and robustness by coarsening in resolution and channel dimensionscited by 2
  6. 2023FV-MgNet: Fully Connected V-cycle MgNet for Interpretable Time Series Forecastingcited by 2