# Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine arXiv: https://arxiv.org/abs/2204.11375 ยท HTML: https://mlsanity.com/paper/2204.11375/ ## Key facts - Cited by 9 papers in the map (data through 14 Oct 2026). - Builds on 20 of its 74 references that are in the map. - First posted on arXiv 24 Apr 2022. ## Builds on (most cited first, 20 of 20) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017) - [Characterizing possible failure modes in physics-informed neural networks](https://mlsanity.com/paper/2109.01050/) (2021) - [When and why PINNs fail to train: A neural tangent kernel perspective](https://mlsanity.com/paper/2007.14527/) (2020) - [DeepXDE: A deep learning library for solving differential equations](https://mlsanity.com/paper/1907.04502/) (2019) - [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017) - [A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks](https://mlsanity.com/paper/2007.07442/) (2020) - [Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2012.02895/) (2020) - [Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equations](https://mlsanity.com/paper/1907.03507/) (2019) - [AdaNet: Adaptive Structural Learning of Artificial Neural Networks](https://mlsanity.com/paper/1607.01097/) (2016) - [Robust Training and Initialization of Deep Neural Networks: An Adaptive Basis Viewpoint](https://mlsanity.com/paper/1912.04862/) (2019) - [Adaptive deep density approximation for Fokker-Planck equations](https://mlsanity.com/paper/2103.11181/) (2021) - [On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements](https://mlsanity.com/paper/2110.14121/) (2021) - [Random Vector Functional Link Neural Network based Ensemble Deep Learning](https://mlsanity.com/paper/1907.00350/) (2019) - [Random Vector Functional Link Networks for Function Approximation on Manifolds](https://mlsanity.com/paper/2007.15776/) (2020) - [A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines](https://mlsanity.com/paper/2103.08042/) (2021) - [VAE-KRnet and its applications to variational Bayes](https://mlsanity.com/paper/2006.16431/) (2020) - [Numerical Approximation of Partial Differential Equations by a Variable Projection Method with Artificial Neural Networks](https://mlsanity.com/paper/2201.09989/) (2022) - [Growing axons: greedy learning of neural networks with application to function approximation](https://mlsanity.com/paper/1910.12686/) (2019) - [A Roadmap for Discretely Energy-Stable Schemes for Dissipative Systems Based on a Generalized Auxiliary Variable with Guaranteed Positivity](https://mlsanity.com/paper/1904.00141/) (2019) ## Built on by (most cited first, 9 of 9) - [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024) - [An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions](https://mlsanity.com/paper/2309.07049/) (2023) - [A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks](https://mlsanity.com/paper/2210.04338/) (2022) - [A Natural Primal-Dual Hybrid Gradient Method for Adversarial Neural Network Training on Solving Partial Differential Equations](https://mlsanity.com/paper/2411.06278/) (2024) - [Error Analysis and Numerical Algorithm for PDE Approximation with Hidden-Layer Concatenated Physics Informed Neural Networks](https://mlsanity.com/paper/2406.06350/) (2024) - [A Novel Tensor Product-Based Neural Network for Solving Partial Differential Equations](https://mlsanity.com/paper/2605.29688/) (2026) - [Multi-Scale Separable Fourier Neural Networks for Solving High-Frequency PDEs](https://mlsanity.com/paper/2605.31027/) (2026) - [Frequency Shift Physics-Informed Extreme Learning Machine for Solving High-Frequency Partial Differential Equations](https://mlsanity.com/paper/2607.01694/) (2026) - [A Novel Fourier Feature Network for Solving Partial Differential Equations](https://mlsanity.com/paper/2608.14733/) (2026) ## Abstract (arXiv) The extreme learning machine (ELM) method can yield highly accurate solutions to linear/nonlinear partial differential equations (PDEs), but requires the last hidden layer of the neural network to be wide to achieve a high accuracy. If the last hidden layer is narrow, the accuracy of the existing ELM method will be poor, irrespective of the rest of the network configuration. In this paper we present a modified ELM method, termed HLConcELM (hidden-layer concatenated ELM), to overcome the above drawback of the conventional ELM method. The HLConcELM method can produce highly accurate solutions to linear/nonlinear PDEs when the last hidden layer of the network is narrow and when it is wide. The new method is based on a type of modified feedforward neural networks (FNN), termed HLConcFNN (hidden-layer concatenated FNN), which incorporates a logical concatenation of the hidden layers in the network and exposes all the hidden nodes to the output-layer nodes. HLConcFNNs have the interesting property that, given a network architecture, when additional hidden layers are appended to the network or when extra nodes are added to the existing hidden layers the representation capacity of the HLConcFNN associated with the new architecture is guaranteed to be not smaller than that of the original network architecture. Here representation capacity refers to the set of all functions that can be exactly represented by the neural network of a given architecture. We present ample benchmark tests with linear/nonlinear PDEs to demonstrate the computational accuracy and performance of the HLConcELM method and the superiority of this method to the conventional ELM from previous works.