# Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search

arXiv: https://arxiv.org/abs/2302.11223 · HTML: https://mlsanity.com/paper/2302.11223/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 60 references that are in the map.
- First posted on arXiv 22 Feb 2023.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Generative Language Modeling for Automated Theorem Proving](https://mlsanity.com/paper/2009.03393/) (2020)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [HyperTree Proof Search for Neural Theorem Proving](https://mlsanity.com/paper/2205.11491/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [Symbolic Regression is NP-hard](https://mlsanity.com/paper/2207.01018/) (2022)
- [Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search](https://mlsanity.com/paper/2205.13134/) (2022)
- [Fast, accurate, and transferable many-body interatomic potentials by symbolic regression](https://mlsanity.com/paper/1904.01095/) (2019)
- [Evolving symbolic density functionals](https://mlsanity.com/paper/2203.02540/) (2022)
- [Learning a Formula of Interpretability to Learn Interpretable Formulas](https://mlsanity.com/paper/2004.11170/) (2020)
- [Complexity Measures for Multi-objective Symbolic Regression](https://mlsanity.com/paper/2109.00238/) (2021)

## Built on by (most cited first, 30 of 41)

- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)
- [R&D-Agent-Quant: A Multi-Agent Framework for Data-Centric Factors and Model Joint Optimization](https://mlsanity.com/paper/2505.15155/) (2025)
- [Automated Scientific Discovery: From Equation Discovery to Autonomous Discovery Systems](https://mlsanity.com/paper/2305.02251/) (2023)
- [Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length](https://mlsanity.com/paper/2411.03753/) (2024)
- [LLM4ED: Large Language Models for Automatic Equation Discovery](https://mlsanity.com/paper/2405.07761/) (2024)
- [Generative Discovery of Partial Differential Equations by Learning from Math Handbooks](https://mlsanity.com/paper/2505.05869/) (2025)
- [A Neural Symbolic Model for Space Physics](https://mlsanity.com/paper/2503.07994/) (2025)
- [Discovering Mathematical Formulas from Data via GPT-guided Monte Carlo Tree Search](https://mlsanity.com/paper/2401.14424/) (2024)
- [Learning to Compute Gr"obner Bases](https://mlsanity.com/paper/2311.12904/) (2023)
- [Vertical Symbolic Regression via Deep Policy Gradient](https://mlsanity.com/paper/2402.00254/) (2024)
- [Grammar-based Ordinary Differential Equation Discovery](https://mlsanity.com/paper/2504.02630/) (2025)
- [Improving Monte Carlo Tree Search for Symbolic Regression](https://mlsanity.com/paper/2509.15929/) (2025)
- [ChatSR: Multimodal Large Language Models for Scientific Formula Discovery](https://mlsanity.com/paper/2406.05410/) (2024)
- [Reconstructing S-matrix Phases with Machine Learning](https://mlsanity.com/paper/2308.09451/) (2023)
- [Complexity-Aware Deep Symbolic Regression with Robust Risk-Seeking Policy Gradients](https://mlsanity.com/paper/2406.06751/) (2024)
- [Diffusion-Based Symbolic Regression](https://mlsanity.com/paper/2505.24776/) (2025)
- [Structured Kolmogorov-Arnold Neural ODEs for Interpretable Learning and Symbolic Discovery of Nonlinear Dynamics](https://mlsanity.com/paper/2506.18339/) (2025)
- [Generative Pre-Trained Transformer for Symbolic Regression Base In-Context Reinforcement Learning](https://mlsanity.com/paper/2404.06330/) (2024)
- [Active Symbolic Discovery of Ordinary Differential Equations via Phase Portrait Sketching](https://mlsanity.com/paper/2409.01416/) (2024)
- [Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large p](https://mlsanity.com/paper/2410.13681/) (2024)
- [Analyzing Generalization in Pre-Trained Symbolic Regression](https://mlsanity.com/paper/2509.19849/) (2025)
- [Alpha2: Discovering Logical Formulaic Alphas using Deep Reinforcement Learning](https://mlsanity.com/paper/2406.16505/) (2024)
- [Sparse Interpretable Deep Learning with LIES Networks for Symbolic Regression](https://mlsanity.com/paper/2506.08267/) (2025)
- [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025)
- [EGG-SR: Embedding Symbolic Equivalence into Symbolic Regression via Equality Graph](https://mlsanity.com/paper/2511.05849/) (2025)
- [Beyond empirical models: Discovering new constitutive laws in solids with graph-based equation discovery](https://mlsanity.com/paper/2511.09906/) (2025)
- [Neuro-Symbolic ODE Discovery with Latent Grammar Flow](https://mlsanity.com/paper/2604.16232/) (2026)
- [Vertical Symbolic Regression](https://mlsanity.com/paper/2312.11955/) (2023)
- [Analytical Lyapunov Function Discovery: An RL-based Generative Approach](https://mlsanity.com/paper/2502.02014/) (2025)
- [MatRL: Provably Generalizable Iterative Algorithm Discovery via Monte-Carlo Tree Search](https://mlsanity.com/paper/2507.03833/) (2025)

## Abstract (arXiv)

Symbolic regression (SR) is the problem of learning a symbolic expression from numerical data. Recently, deep neural models trained on procedurally-generated synthetic datasets showed competitive performance compared to more classical Genetic Programming (GP) algorithms. Unlike their GP counterparts, these neural approaches are trained to generate expressions from datasets given as context. This allows them to produce accurate expressions in a single forward pass at test time. However, they usually do not benefit from search abilities, which result in low performance compared to GP on out-of-distribution datasets. In this paper, we propose a novel method which provides the best of both worlds, based on a Monte-Carlo Tree Search procedure using a context-aware neural mutation model, which is initially pre-trained to learn promising mutations, and further refined from successful experiences in an online fashion. The approach demonstrates state-of-the-art performance on the well-known textttSRBench benchmark.