# Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression arXiv: https://arxiv.org/abs/2302.00773 ยท HTML: https://mlsanity.com/paper/2302.00773/ ## Key facts - Cited by 14 papers in the map (data through 14 Oct 2026). - Builds on 19 of its 43 references that are in the map. - First posted on arXiv 1 Feb 2023. ## Builds on (most cited first, 19 of 19) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019) - [Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systems](https://mlsanity.com/paper/1903.12394/) (2019) - [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021) - [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018) - [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016) - [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021) - [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019) - [SymFormer: End-to-end symbolic regression using transformer-based architecture](https://mlsanity.com/paper/2205.15764/) (2022) - [Deep Symbolic Regression for Recurrent Sequences](https://mlsanity.com/paper/2201.04600/) (2022) - [Shape-constrained Symbolic Regression -- Improving Extrapolation with Prior Knowledge](https://mlsanity.com/paper/2103.15624/) (2021) - [OccamNet: A Fast Neural Model for Symbolic Regression at Scale](https://mlsanity.com/paper/2007.10784/) (2020) - [Symbolic Regression Driven by Training Data and Prior Knowledge](https://mlsanity.com/paper/2004.11947/) (2020) - [Symbolic regression outperforms other models for small data sets](https://mlsanity.com/paper/2103.15147/) (2021) - [Informed Equation Learning](https://mlsanity.com/paper/2105.06331/) (2021) - [Constructing Parsimonious Analytic Models for Dynamic Systems via Symbolic Regression](https://mlsanity.com/paper/1903.11483/) (2019) - [Logic Guided Genetic Algorithms](https://mlsanity.com/paper/2010.11328/) (2020) - [Bayesian Learning to Discover Mathematical Operations in Governing Equations of Dynamic Systems](https://mlsanity.com/paper/2206.00669/) (2022) ## Built on by (most cited first, 14 of 14) - [Automated Scientific Discovery: From Equation Discovery to Autonomous Discovery Systems](https://mlsanity.com/paper/2305.02251/) (2023) - [Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length](https://mlsanity.com/paper/2411.03753/) (2024) - [Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods](https://mlsanity.com/paper/2410.05592/) (2024) - [Neuro-Symbolic AI: Explainability, Challenges, and Future Trends](https://mlsanity.com/paper/2411.04383/) (2024) - [SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression](https://mlsanity.com/paper/2401.09949/) (2024) - [Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Models](https://mlsanity.com/paper/2509.03036/) (2025) - [Bayesian polynomial neural networks and polynomial neural ordinary differential equations](https://mlsanity.com/paper/2308.10892/) (2023) - [Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methods](https://mlsanity.com/paper/2412.01181/) (2024) - [Machine learning-guided construction of an analytic kinetic energy functional for orbital free density functional theory](https://mlsanity.com/paper/2502.05411/) (2025) - [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025) - [A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression](https://mlsanity.com/paper/2407.00065/) (2024) - [Baseflow identification via explainable AI with Kolmogorov-Arnold networks](https://mlsanity.com/paper/2410.11587/) (2024) - [Neuro-Evolutionary Approach to Physics-Aware Symbolic Regression](https://mlsanity.com/paper/2504.16503/) (2025) - [Gaussian Process Regression -- Neural Network Hybrid with Optimized Redundant Coordinates](https://mlsanity.com/paper/2509.08457/) (2025) ## Abstract (arXiv) Many real-world systems can be described by mathematical models that are human-comprehensible, easy to analyze and help explain the system's behavior. Symbolic regression is a method that can automatically generate such models from data. Historically, symbolic regression has been predominantly realized by genetic programming, a method that evolves populations of candidate solutions that are subsequently modified by genetic operators crossover and mutation. However, this approach suffers from several deficiencies: it does not scale well with the number of variables and samples in the training data - models tend to grow in size and complexity without an adequate accuracy gain, and it is hard to fine-tune the model coefficients using just genetic operators. Recently, neural networks have been applied to learn the whole analytic model, i.e., its structure and the coefficients, using gradient-based optimization algorithms. This paper proposes a novel neural network-based symbolic regression method that constructs physically plausible models based on even very small training data sets and prior knowledge about the system. The method employs an adaptive weighting scheme to effectively deal with multiple loss function terms and an epoch-wise learning process to reduce the chance of getting stuck in poor local optima. Furthermore, we propose a parameter-free method for choosing the model with the best interpolation and extrapolation performance out of all the models generated throughout the whole learning process. We experimentally evaluate the approach on four test systems: the TurtleBot 2 mobile robot, the magnetic manipulation system, the equivalent resistance of two resistors in parallel, and the longitudinal force of the anti-lock braking system. The results clearly show the potential of the method to find parsimonious models that comply with the prior knowledge provided.