# Deep Generative Symbolic Regression

arXiv: https://arxiv.org/abs/2401.00282 · HTML: https://mlsanity.com/paper/2401.00282/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 57 references that are in the map.
- First posted on arXiv 30 Dec 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [A Distributional Perspective on Reinforcement Learning](https://mlsanity.com/paper/1707.06887/) (2017)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [Deep Neuroevolution: Genetic Algorithms Are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning](https://mlsanity.com/paper/1712.06567/) (2017)
- [Reinforcement Learning for Integer Programming: Learning to Cut](https://mlsanity.com/paper/1906.04859/) (2019)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [PMLB v1.0: An open source dataset collection for benchmarking machine learning methods](https://mlsanity.com/paper/2012.00058/) (2020)
- [Meta-Learning Priors for Efficient Online Bayesian Regression](https://mlsanity.com/paper/1807.08912/) (2018)
- [Discovering Sparse Interpretable Dynamics from Partial Observations](https://mlsanity.com/paper/2107.10879/) (2021)
- [Neural Program Synthesis with Priority Queue Training](https://mlsanity.com/paper/1801.03526/) (2018)
- [Learning outside the Black-Box: The pursuit of interpretable models](https://mlsanity.com/paper/2011.08596/) (2020)
- [Genetic-Gated Networks for Deep Reinforcement](https://mlsanity.com/paper/1903.01886/) (2018)
- [AI-Assisted Discovery of Quantitative and Formal Models in Social Science](https://mlsanity.com/paper/2210.00563/) (2022)
- [RealPoint3D: Point Cloud Generation from a Single Image with Complex Background](https://mlsanity.com/paper/1809.02743/) (2018)

## Built on by (most cited first, 28 of 28)

- [Discovering Preference Optimization Algorithms with and for Large Language Models](https://mlsanity.com/paper/2406.08414/) (2024)
- [A Comprehensive Survey on Self-Interpretable Neural Networks](https://mlsanity.com/paper/2501.15638/) (2025)
- [Automatically Learning Hybrid Digital Twins of Dynamical Systems](https://mlsanity.com/paper/2410.23691/) (2024)
- [MMSR: Symbolic Regression is a Multi-Modal Information Fusion Task](https://mlsanity.com/paper/2402.18603/) (2024)
- [ODE Discovery for Longitudinal Heterogeneous Treatment Effects Inference](https://mlsanity.com/paper/2403.10766/) (2024)
- [GENSR: Symbolic Regression Based in Equation Generative Space](https://mlsanity.com/paper/2602.20557/) (2026)
- [Discovering Symbolic Differential Equations with Symmetry Invariants](https://mlsanity.com/paper/2505.12083/) (2025)
- [When is a System Discoverable from Data? Discovery Requires Chaos](https://mlsanity.com/paper/2511.08860/) (2025)
- [A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programming](https://mlsanity.com/paper/2406.03585/) (2024)
- [Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients](https://mlsanity.com/paper/2406.06751/) (2024)
- [Bridging the Domain Gap in Equation Distillation with Reinforcement Feedback](https://mlsanity.com/paper/2505.15572/) (2025)
- [Diffusion-Based Symbolic Regression](https://mlsanity.com/paper/2505.24776/) (2025)
- [Generalizing the SINDy approach with nested neural networks](https://mlsanity.com/paper/2404.15742/) (2024)
- [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025)
- [Scheduled Knowledge Acquisition on Lightweight Vector Symbolic Architectures for Brain-Computer Interfaces](https://mlsanity.com/paper/2403.13844/) (2024)
- [Shape Arithmetic Expressions: Advancing Scientific Discovery Beyond Closed-Form Equations](https://mlsanity.com/paper/2404.09788/) (2024)
- [DN-CL: Deep Symbolic Regression against Noise via Contrastive Learning](https://mlsanity.com/paper/2406.14844/) (2024)
- [Analytical Lyapunov Function Discovery: An RL-based Generative Approach](https://mlsanity.com/paper/2502.02014/) (2025)
- [Symmetry-Constrained Language-Guided Program Synthesis for Discovering Governing Equations from Noisy and Partial Observations](https://mlsanity.com/paper/2603.06869/) (2026)
- [Symbolic Density Estimation for Discrete Distributions](https://mlsanity.com/paper/2605.21813/) (2026)
- [SRLR: Symbolic Regression based Logic Recovery to Counter Programmable Logic Controller Attacks](https://mlsanity.com/paper/2512.11298/) (2025)
- [ECSEL: Explainable Classification via Signomial Equation Learning](https://mlsanity.com/paper/2601.21789/) (2026)
- [Knowledge-Informed Kernel State Reconstruction from Heterogeneous Partial Observations](https://mlsanity.com/paper/2601.22328/) (2026)
- [Explaining the Explainer: Understanding the Inner Workings of Transformer-based Symbolic Regression Models](https://mlsanity.com/paper/2602.03506/) (2026)
- [Symbolic recovery of PDEs from measurement data](https://mlsanity.com/paper/2602.15603/) (2026)
- [Deep Variational Inference Symbolic Regression](https://mlsanity.com/paper/2605.01067/) (2026)
- [Decision-Making under Combinatorial Risk](https://mlsanity.com/paper/2606.10092/) (2026)
- [DASS: A Solver-Adaptable Dynamic Auxiliary Search Strategy for Symbolic Regression](https://mlsanity.com/paper/2610.16812/) (2026)

## Abstract (arXiv)

Symbolic regression (SR) aims to discover concise closed-form mathematical equations from data, a task fundamental to scientific discovery. However, the problem is highly challenging because closed-form equations lie in a complex combinatorial search space. Existing methods, ranging from heuristic search to reinforcement learning, fail to scale with the number of input variables. We make the observation that closed-form equations often have structural characteristics and invariances (e.g., the commutative law) that could be further exploited to build more effective symbolic regression solutions. Motivated by this observation, our key contribution is to leverage pre-trained deep generative models to capture the intrinsic regularities of equations, thereby providing a solid foundation for subsequent optimization steps. We show that our novel formalism unifies several prominent approaches of symbolic regression and offers a new perspective to justify and improve on the previous ad hoc designs, such as the usage of cross-entropy loss during pre-training. Specifically, we propose an instantiation of our framework, Deep Generative Symbolic Regression (DGSR). In our experiments, we show that DGSR achieves a higher recovery rate of true equations in the setting of a larger number of input variables, and it is more computationally efficient at inference time than state-of-the-art RL symbolic regression solutions.