mlsanityData to 14 Oct 2026

Computational Advantages of Multi-Grade Deep Learning: Convergence Analysis and Performance Insights

arXiv 2507.20351 ↗First posted 27 Jul 2025cs.LG cs.NA math.NAcited by 3

Not on the map. Why?

Abstract (arXiv)

Multi-grade deep learning (MGDL) has been shown to significantly outperform the standard single-grade deep learning (SGDL) across various applications. This work aims to investigate the computational advantages of MGDL focusing on its performance in image regression, denoising, and deblurring tasks, and comparing it to SGDL. We establish convergence results for the gradient descent (GD) method applied to these models and provide mathematical insights into MGDL's improved performance. In particular, we demonstrate that MGDL is more robust to the choice of learning rate under GD than SGDL. Furthermore, we analyze the eigenvalue distributions of the Jacobian matrices associated with the iterative schemes arising from the GD iterations, offering an explanation for MGDL's enhanced training stability.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 18 of its 38 references that are in the map. First posted on arXiv 27 Jul 2025.

Builds on

18 of 38 references.

References · 18

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2018On the Spectral Bias of Neural Networkscited by 1,341
  6. 2021Gradient Descent on Neural Networks Typically Occurs at the Edge of Stabilitycited by 427
  7. 2018Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Networkcited by 420
  8. 2018Training behavior of deep neural network in frequency domaincited by 174
  9. 2020Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networkscited by 98
  10. 2020Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domainscited by 79
  11. 2020Convex Geometry and Duality of Over-parameterized Neural Networkscited by 49
  12. 2024Addressing Spectral Bias of Deep Neural Networks by Multi-Grade Deep Learningcited by 14
6 more
  1. 2023Multi-Grade Deep Learningcited by 12
  2. 2023Multi-Grade Deep Learning for Partial Differential Equations with Applications to the Burgers Equationcited by 5
  3. 2024Deep Neural Network Solutions for Oscillatory Fredholm Integral Equationscited by 5
  4. 2023Successive Affine Learning for Deep Neural Networkscited by 4
  5. 2024Inexact FPPA for the ℓ0 Sparse Regularization Problemcited by 3
  6. 2021A Fast Convergent Ordered-Subsets Algorithm with Subiteration-Dependent Preconditioners for PET Image Reconstructioncited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026Multigrade Neural Network Approximationcited by 2
  2. 2026Layer-wise Geometric Approximation Rates for Deep Networkscited by 2
  3. 2026MG-SpaIR: Multi-grade Sparse-guided Implicit Representation for Training-Data-Free Image Restorationcited by 0