# A Modified Batch Intrinsic Plasticity Method for Pre-training the Random Coefficients of Extreme Learning Machines arXiv: https://arxiv.org/abs/2103.08042 ยท HTML: https://mlsanity.com/paper/2103.08042/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 10 of its 54 references that are in the map. - First posted on arXiv 14 Mar 2021. ## Builds on (most cited first, 10 of 10) - [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017) - [DGM: A deep learning algorithm for solving partial differential equations](https://mlsanity.com/paper/1708.07469/) (2017) - [The Deep Ritz method: A deep learning-based numerical algorithm for solving variational problems](https://mlsanity.com/paper/1710.00211/) (2017) - [An Energy Approach to the Solution of Partial Differential Equations in Computational Mechanics via Machine Learning: Concepts, Implementation and Applications](https://mlsanity.com/paper/1908.10407/) (2019) - [A Method for Representing Periodic Functions and Enforcing Exactly Periodic Boundary Conditions with Deep Neural Networks](https://mlsanity.com/paper/2007.07442/) (2020) - [Local Extreme Learning Machines and Domain Decomposition for Solving Linear and Nonlinear Partial Differential Equations](https://mlsanity.com/paper/2012.02895/) (2020) - [Physics Informed Extreme Learning Machine (PIELM) -- A rapid method for the numerical solution of partial differential equations](https://mlsanity.com/paper/1907.03507/) (2019) - [Stochastic Configuration Networks: Fundamentals and Algorithms](https://mlsanity.com/paper/1702.03180/) (2017) - [A Method of Generating Random Weights and Biases in Feedforward Neural Networks with Random Hidden Nodes](https://mlsanity.com/paper/1710.04874/) (2017) - [Deep Coregionalization for the Emulation of Spatial-Temporal Fields](https://mlsanity.com/paper/1910.07577/) (2019) ## Built on by (most cited first, 14 of 14) - [Weak baselines and reporting biases lead to overoptimism in machine learning for fluid-related partial differential equations](https://mlsanity.com/paper/2407.07218/) (2024) - [On Computing the Hyperparameter of Extreme Learning Machines: Algorithm and Application to Computational PDEs, and Comparison with Classical and High-Order Finite Elements](https://mlsanity.com/paper/2110.14121/) (2021) - [An Extreme Learning Machine-Based Method for Computational PDEs in Higher Dimensions](https://mlsanity.com/paper/2309.07049/) (2023) - [Parsimonious Physics-Informed Random Projection Neural Networks for Initial-Value Problems of ODEs and index-1 DAEs](https://mlsanity.com/paper/2203.05337/) (2022) - [A Method for Computing Inverse Parametric PDE Problems with Random-Weight Neural Networks](https://mlsanity.com/paper/2210.04338/) (2022) - [Numerical Approximation of Partial Differential Equations by a Variable Projection Method with Artificial Neural Networks](https://mlsanity.com/paper/2201.09989/) (2022) - [Numerical Computation of Partial Differential Equations by Hidden-Layer Concatenated Extreme Learning Machine](https://mlsanity.com/paper/2204.11375/) (2022) - [Numerical Solution of Stiff ODEs with Physics-Informed RPNNs](https://mlsanity.com/paper/2108.01584/) (2021) - [Error Analysis of Physics-Informed Neural Networks for Approximating Dynamic PDEs of Second Order in Time](https://mlsanity.com/paper/2303.12245/) (2023) - [Fredholm Neural Networks for inverse problems in elliptic PDEs](https://mlsanity.com/paper/2507.06038/) (2025) - [HEATNETs: Explainable Random Feature Neural Networks for High-Dimensional Parabolic PDEs](https://mlsanity.com/paper/2511.00886/) (2025) - [A Novel Method for Enforcing Exactly Dirichlet, Neumann and Robin Conditions on Curved Domain Boundaries for Physics Informed Machine Learning](https://mlsanity.com/paper/2603.21909/) (2026) - [Error Analysis and Numerical Algorithm for PDE Approximation with Hidden-Layer Concatenated Physics Informed Neural Networks](https://mlsanity.com/paper/2406.06350/) (2024) - [Exact Constraint Enforcement in Physics-Informed Extreme Learning Machines using Null-Space Projection Framework](https://mlsanity.com/paper/2601.10999/) (2026) ## Abstract (arXiv) In extreme learning machines (ELM) the hidden-layer coefficients are randomly set and fixed, while the output-layer coefficients of the neural network are computed by a least squares method. The randomly-assigned coefficients in ELM are known to influence its performance and accuracy significantly. In this paper we present a modified batch intrinsic plasticity (modBIP) method for pre-training the random coefficients in the ELM neural networks. The current method is devised based on the same principle as the batch intrinsic plasticity (BIP) method, namely, by enhancing the information transmission in every node of the neural network. It differs from BIP in two prominent aspects. First, modBIP does not involve the activation function in its algorithm, and it can be applied with any activation function in the neural network. In contrast, BIP employs the inverse of the activation function in its construction, and requires the activation function to be invertible (or monotonic). The modBIP method can work with the often-used non-monotonic activation functions (e.g. Gaussian, swish, Gaussian error linear unit, and radial-basis type functions), with which BIP breaks down. Second, modBIP generates target samples on random intervals with a minimum size, which leads to highly accurate computation results when combined with ELM. The combined ELM/modBIP method is markedly more accurate than ELM/BIP in numerical simulations. Ample numerical experiments are presented with shallow and deep neural networks for function approximation and boundary/initial value problems with partial differential equations. They demonstrate that the combined ELM/modBIP method produces highly accurate simulation results, and that its accuracy is insensitive to the random-coefficient initializations in the neural network. This is in sharp contrast with the ELM results without pre-training of the random coefficients.