mlsanityData to 7 Oct 2026

A Nonoverlapping Domain Decomposition Method for Extreme Learning Machines: Elliptic Problems

arXiv 2406.15959 ↗First posted 22 Jun 2024math.NA cs.CE cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Extreme learning machine (ELM) is a methodology for solving partial differential equations (PDEs) using a single hidden layer feed-forward neural network. It presets the weight/bias coefficients in the hidden layer with random values, which remain fixed throughout the computation, and uses a linear least squares method for training the parameters of the output layer of the neural network. It is known to be much faster than Physics informed neural networks. However, classical ELM is still computationally expensive when a high level of representation is desired in the solution as this requires solving a large least squares system. In this paper, we propose a nonoverlapping domain decomposition method (DDM) for ELMs that not only reduces the training time of ELMs, but is also suitable for parallel computation. In numerical analysis, DDMs have been widely studied to reduce the time to obtain finite element solutions for elliptic PDEs through parallel computation. Among these approaches, nonoverlapping DDMs are attracting the most attention. Motivated by these methods, we introduce local neural networks, which are valid only at corresponding subdomains, and an auxiliary variable at the interface. We construct a system on the variable and the parameters of local neural networks. A Schur complement system on the interface can be derived by eliminating the parameters of the output layer. The auxiliary variable is then directly obtained by solving the reduced system after which the parameters for each local neural network are solved in parallel. A method for initializing the hidden layer parameters suitable for high approximation quality in large systems is also proposed. Numerical results that verify the acceleration performance of the proposed method with respect to the number of subdomains are presented.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 20 of its 67 references that are in the map. First posted on arXiv 22 Jun 2024.

Builds on

20 of 67 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  5. 2015Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classificationcited by 3,590
  6. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  7. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  8. 2018On the Spectral Bias of Neural Networkscited by 1,321
  9. 2020When and why PINNs fail to train: A neural tangent kernel perspectivecited by 629
  10. 2021Finite Basis Physics-Informed Neural Networks (FBPINNs): a scalable domain decomposition approach for solving differential equationscited by 142
  11. 2020Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equationscited by 78
  12. 2019Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equationscited by 66
8 more
  1. 2018Layer-Parallel Training of Deep Residual Neural Networkscited by 35
  2. 2023An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensionscited by 21
  3. 2023Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategiescited by 20
  4. 2021A coarse space acceleration of deep-DDMcited by 7
  5. 2023A Block-Coordinate Approach of Multi-level Optimization with an Application to Physics-Informed Neural Networkscited by 7
  6. 2019Additive Schwarz Methods for Convex Optimization as Gradient Methodscited by 4
  7. 2021Parareal Neural Networks Emulating a Parallel-in-time Algorithmcited by 4
  8. 2021Two-level Group Convolutioncited by 4

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 3.

4 papers.

  1. 2025Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Methodcited by 6
  2. 2024Are Two Hidden Layers Still Enough for the Physics-Informed Neural Networks?cited by 3
  3. 2025Orthogonal greedy algorithm for linear operator learning with shallow neural networkcited by 3
  4. 2025A Neumann-Neumann Acceleration with Coarse Space for Domain Decomposition of Extreme Learning Machinescited by 0