mlsanityData to 7 Oct 2026

Learn Singularly Perturbed Solutions via Homotopy Dynamics

arXiv 2502.00488 ↗First posted 1 Feb 2025cs.LG cs.NA math.NAcited by 2

Not on the map. Why?

Abstract (arXiv)

Solving partial differential equations (PDEs) using neural networks has become a central focus in scientific machine learning. Training neural networks for singularly perturbed problems is particularly challenging due to certain parameters in the PDEs that introduce near-singularities in the loss function. In this study, we overcome this challenge by introducing a novel method based on homotopy dynamics to effectively manipulate these parameters. From a theoretical perspective, we analyze the effects of these parameters on training difficulty in these singularly perturbed problems and establish the convergence of the proposed homotopy dynamics method. Experimentally, we demonstrate that our approach significantly accelerates convergence and improves the accuracy of these singularly perturbed problems. These findings present an efficient optimization strategy leveraging homotopy dynamics, offering a robust framework to extend the applicability of neural networks for solving singularly perturbed differential equations.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 33 of its 62 references that are in the map. First posted on arXiv 1 Feb 2025.

Builds on

33 of 62 references.

References · 33

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  2. 2020Fourier Neural Operator for Parametric Partial Differential Equationscited by 2,806
  3. 2019DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operatorscited by 1,965
  4. 2018A Convergence Theory for Deep Learning via Over-Parameterizationcited by 784
  5. 2021Characterizing possible failure modes in physics-informed neural networkscited by 632
  6. 2022Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's nextcited by 603
  7. 2017The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problemscited by 581
  8. 2018Gradient Descent Finds Global Minima of Deep Neural Networkscited by 572
  9. 2019On Exact Computation with an Infinitely Wide Neural Netcited by 572
  10. 2017Solving high-dimensional partial differential equations using deep learningcited by 549
  11. 2020On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networkscited by 279
  12. 2020Solving Allen-Cahn and Cahn-Hilliard Equations using the Adaptive Physics Informed Neural Networkscited by 122
21 more
  1. 2024Challenges in Training PINNs: A Loss Landscape Perspectivecited by 109
  2. 2020Multi-scale Deep Neural Network (MscaleDNN) for Solving Poisson-Boltzmann Equation in Complex Domainscited by 78
  3. 2022Overview frequency principle/spectral bias in deep learningcited by 68
  4. 2019Generalization Error Bounds of Gradient Descent for Learning Over-parameterized Deep ReLU Networkscited by 60
  5. 2022Deep Nonparametric Estimation of Operators between Infinite Dimensional Spacescited by 45
  6. 2022Mitigating spectral bias for the multiscale operator learningcited by 43
  7. 2020Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theorycited by 41
  8. 2023MgNO: Efficient Parameterization of Linear Operators via Multigridcited by 40
  9. 2023Phase-Field DeepONet: Physics-informed deep operator neural network for fast simulations of pattern formation governed by gradient flows of free-energy functionalscited by 26
  10. 2022A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networkscited by 14
  11. 2023Energy-Dissipative Evolutionary Deep Operator Neural Networkscited by 14
  12. 2024Deeper or Wider: A Perspective from Optimal Generalization Error with Sobolev Losscited by 14
  13. 2023Failure-informed adaptive sampling for PINNs, Part II: combining with re-sampling and subset simulationcited by 13
  14. 2022DOSnet as a Non-Black-Box PDE Solver: When Deep Learning Meets Operator Splittingcited by 11
  15. 2024Automatic Differentiation is Essential in Training Neural Networks for Solving Differential Equationscited by 9
  16. 2022Approximation of Functionals by Neural Network without Curse of Dimensionalitycited by 8
  17. 2019A homotopy training algorithm for fully connected neural networkscited by 7
  18. 2024Frequency-adaptive Multi-scale Deep Neural Networkscited by 5
  19. 2020Towards an Understanding of Residual Networks Using Neural Tangent Hierarchy (NTH)cited by 4
  20. 2023Homotopy Relaxation Training Algorithms for Infinite-Width Two-Layer ReLU Neural Networkscited by 4
  21. 2024Quantifying Training Difficulty and Accelerating Convergence in Neural Network-Based PDE Solverscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025ZENN: A Thermodynamics-Inspired Computational Framework for Heterogeneous Data-Driven Modelingcited by 1
  2. 2026Discontinuous Galerkin finite element operator network for solving non-smooth PDEscited by 1