# Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery

arXiv: https://arxiv.org/abs/1912.04825 · HTML: https://mlsanity.com/paper/1912.04825/

## Key facts

- Cited by 93 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Dec 2019.

## Built on by (most cited first, 30 of 93)

- [The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks](https://mlsanity.com/paper/2306.17844/) (2023)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search](https://mlsanity.com/paper/2205.13134/) (2022)
- [Deep Symbolic Regression for Recurrent Sequences](https://mlsanity.com/paper/2201.04600/) (2022)
- [Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery](https://mlsanity.com/paper/2206.10540/) (2022)
- [Discovering Causal Relations and Equations from Data](https://mlsanity.com/paper/2305.13341/) (2023)
- [Shape-constrained Symbolic Regression -- Improving Extrapolation with Prior Knowledge](https://mlsanity.com/paper/2103.15624/) (2021)
- [SINDy-RL: Interpretable and Efficient Model-Based Reinforcement Learning](https://mlsanity.com/paper/2403.09110/) (2024)
- [Discovering Sparse Interpretable Dynamics from Partial Observations](https://mlsanity.com/paper/2107.10879/) (2021)
- [Discovering Nonlinear PDEs from Scarce Data with Physics-encoded Learning](https://mlsanity.com/paper/2201.12354/) (2022)
- [Deep Learning and Symbolic Regression for Discovering Parametric Equations](https://mlsanity.com/paper/2207.00529/) (2022)
- [DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience](https://mlsanity.com/paper/2506.04282/) (2025)
- [An Approach to Symbolic Regression Using Feyn](https://mlsanity.com/paper/2104.05417/) (2021)
- [Taylor Genetic Programming for Symbolic Regression](https://mlsanity.com/paper/2205.09751/) (2022)
- [Physics-informed Spline Learning for Nonlinear Dynamics Discovery](https://mlsanity.com/paper/2105.02368/) (2021)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length](https://mlsanity.com/paper/2411.03753/) (2024)
- [Symbolic regression for scientific discovery: an application to wind speed forecasting](https://mlsanity.com/paper/2102.10570/) (2021)
- [A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data](https://mlsanity.com/paper/2309.13705/) (2023)
- [Extreme sparsification of physics-augmented neural networks for interpretable model discovery in mechanics](https://mlsanity.com/paper/2310.03652/) (2023)
- [Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression](https://mlsanity.com/paper/2302.00773/) (2023)
- [Interpretable Machine Learning in Physics: A Review](https://mlsanity.com/paper/2503.23616/) (2025)
- [Bayesian Spline Learning for Equation Discovery of Nonlinear Dynamics with Quantified Uncertainty](https://mlsanity.com/paper/2210.08095/) (2022)
- [MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task](https://mlsanity.com/paper/2402.18603/) (2024)
- [Symbolic regression outperforms other models for small data sets](https://mlsanity.com/paper/2103.15147/) (2021)
- [Koopman operator learning using invertible neural networks](https://mlsanity.com/paper/2306.17396/) (2023)
- [GSR: A Generalized Symbolic Regression Approach](https://mlsanity.com/paper/2205.15569/) (2022)
- [Physics-aware Machine Learning Revolutionizes Scientific Paradigm for Machine Learning and Process-based Hydrology](https://mlsanity.com/paper/2310.05227/) (2023)
- [Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods](https://mlsanity.com/paper/2410.05592/) (2024)
- [Informed Equation Learning](https://mlsanity.com/paper/2105.06331/) (2021)

## Abstract (arXiv)

Symbolic regression is a powerful technique that can discover analytical equations that describe data, which can lead to explainable models and generalizability outside of the training data set. In contrast, neural networks have achieved amazing levels of accuracy on image recognition and natural language processing tasks, but are often seen as black-box models that are difficult to interpret and typically extrapolate poorly. Here we use a neural network-based architecture for symbolic regression called the Equation Learner (EQL) network and integrate it with other deep learning architectures such that the whole system can be trained end-to-end through backpropagation. To demonstrate the power of such systems, we study their performance on several substantially different tasks. First, we show that the neural network can perform symbolic regression and learn the form of several functions. Next, we present an MNIST arithmetic task where a separate part of the neural network extracts the digits. Finally, we demonstrate prediction of dynamical systems where an unknown parameter is extracted through an encoder. We find that the EQL-based architecture can extrapolate quite well outside of the training data set compared to a standard neural network-based architecture, paving the way for deep learning to be applied in scientific exploration and discovery.