# Deep Learning and Symbolic Regression for Discovering Parametric Equations

arXiv: https://arxiv.org/abs/2207.00529 · HTML: https://mlsanity.com/paper/2207.00529/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 50 references that are in the map.
- First posted on arXiv 1 Jul 2022.

## Builds on (most cited first, 30 of 32)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [HyperNetworks](https://mlsanity.com/paper/1609.09106/) (2016)
- [Meta-Learning in Neural Networks: A Survey](https://mlsanity.com/paper/2004.05439/) (2020)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Learning to learn by gradient descent by gradient descent](https://mlsanity.com/paper/1606.04474/) (2016)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Machine learning accelerated computational fluid dynamics](https://mlsanity.com/paper/2102.01010/) (2021)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Meta Networks](https://mlsanity.com/paper/1703.00837/) (2017)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Functional Variational Bayesian Neural Networks](https://mlsanity.com/paper/1903.05779/) (2019)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Continual Deep Learning by Functional Regularisation of Memorable Past](https://mlsanity.com/paper/2004.14070/) (2020)
- [Neural Networks Fail to Learn Periodic Functions and How to Fix It](https://mlsanity.com/paper/2006.08195/) (2020)
- [Measuring and regularizing networks in function space](https://mlsanity.com/paper/1805.08289/) (2018)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [Neural Arithmetic Logic Units](https://mlsanity.com/paper/1808.00508/) (2018)
- [Pad'e Activation Units: End-to-end Learning of Flexible Activation Functions in Deep Networks](https://mlsanity.com/paper/1907.06732/) (2019)
- [Generalization Error in Deep Learning](https://mlsanity.com/paper/1808.01174/) (2018)
- [Understanding Why Neural Networks Generalize Well Through GSNR of Parameters](https://mlsanity.com/paper/2001.07384/) (2020)
- [HyperPINN: Learning parameterized differential equations with physics-informed hypernetworks](https://mlsanity.com/paper/2111.01008/) (2021)
- [Discovering Sparse Interpretable Dynamics from Partial Observations](https://mlsanity.com/paper/2107.10879/) (2021)
- [Using Machine Learning to Augment Coarse-Grid Computational Fluid Dynamics Simulations](https://mlsanity.com/paper/2010.00072/) (2020)

## Built on by (most cited first, 23 of 23)

- [Training Stiff Neural Ordinary Differential Equations with Implicit Single-Step Methods](https://mlsanity.com/paper/2410.05592/) (2024)
- [BEDI: A Comprehensive Benchmark for Evaluating Embodied Agents on UAVs](https://mlsanity.com/paper/2505.18229/) (2025)
- [Symbolic Regression for PDEs using Pruned Differentiable Programs](https://mlsanity.com/paper/2303.07009/) (2023)
- [Bayesian polynomial neural networks and polynomial neural ordinary differential equations](https://mlsanity.com/paper/2308.10892/) (2023)
- [Training Stiff Neural Ordinary Differential Equations with Explicit Exponential Integration Methods](https://mlsanity.com/paper/2412.01181/) (2024)
- [Bridging the Domain Gap in Equation Distillation with Reinforcement Feedback](https://mlsanity.com/paper/2505.15572/) (2025)
- [No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks](https://mlsanity.com/paper/2405.02369/) (2024)
- [Data-driven discovery of governing differential equations across physical systems](https://mlsanity.com/paper/2606.09638/) (2026)
- [Closed-form Solutions: A New Perspective on Solving Differential Equations](https://mlsanity.com/paper/2405.14620/) (2024)
- [A Neural Operator-Based Approach to Symbolic Discovery of PDEs](https://mlsanity.com/paper/2501.08086/) (2025)
- [Sparse Interpretable Deep Learning with LIES Networks for Symbolic Regression](https://mlsanity.com/paper/2506.08267/) (2025)
- [Automated Discovery of Conservation Laws via Hybrid Neural ODE-Transformers](https://mlsanity.com/paper/2511.00102/) (2025)
- [Learning Physical Operators using Neural Operators](https://mlsanity.com/paper/2602.23113/) (2026)
- [A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression](https://mlsanity.com/paper/2407.00065/) (2024)
- [Interpretable Non-linear Survival Analysis with Evolutionary Symbolic Regression](https://mlsanity.com/paper/2504.05756/) (2025)
- [CTSR: Cartesian tensor-based sparse regression for data-driven discovery of high-dimensional invariant governing equations](https://mlsanity.com/paper/2504.07618/) (2025)
- [Data-Efficient Symbolic Regression via Foundation Model Distillation](https://mlsanity.com/paper/2508.19487/) (2025)
- [Optimized Architectures for Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2512.12448/) (2025)
- [SymPlex: A Structure-Aware Transformer for Symbolic PDE Solving](https://mlsanity.com/paper/2602.03816/) (2026)
- [Identifying Ordinary Differential Equations for Data-efficient Model-based Reinforcement Learning](https://mlsanity.com/paper/2406.19817/) (2024)
- [Deep Variational Inference Symbolic Regression](https://mlsanity.com/paper/2605.01067/) (2026)
- [Hypothesize, Evaluate, Refine: A Scientific Agent for PDE Discovery with Unknown Spatial Coefficient Fields](https://mlsanity.com/paper/2608.27475/) (2026)
- [SMILE: Bridging Continuous Optimization and Discrete Symbolic Recovery](https://mlsanity.com/paper/2609.04639/) (2026)

## Abstract (arXiv)

Symbolic regression is a machine learning technique that can learn the governing formulas of data and thus has the potential to transform scientific discovery. However, symbolic regression is still limited in the complexity and dimensionality of the systems that it can analyze. Deep learning on the other hand has transformed machine learning in its ability to analyze extremely complex and high-dimensional datasets. We propose a neural network architecture to extend symbolic regression to parametric systems where some coefficient may vary but the structure of the underlying governing equation remains constant. We demonstrate our method on various analytic expressions, ODEs, and PDEs with varying coefficients and show that it extrapolates well outside of the training domain. The neural network-based architecture can also integrate with other deep learning architectures so that it can analyze high-dimensional data while being trained end-to-end. To this end we integrate our architecture with convolutional neural networks to analyze 1D images of varying spring systems.