# Differentiable Genetic Programming for High-dimensional Symbolic Regression

arXiv: https://arxiv.org/abs/2304.08915 · HTML: https://mlsanity.com/paper/2304.08915/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 53 references that are in the map.
- First posted on arXiv 18 Apr 2023.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Fair DARTS: Eliminating Unfair Advantages in Differentiable Architecture Search](https://mlsanity.com/paper/1911.12126/) (2019)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [SymFormer: End-to-end symbolic regression using transformer-based architecture](https://mlsanity.com/paper/2205.15764/) (2022)
- [Evolving Deep Convolutional Neural Networks for Image Classification](https://mlsanity.com/paper/1710.10741/) (2017)
- [Where are we now? A large benchmark study of recent symbolic regression methods](https://mlsanity.com/paper/1804.09331/) (2018)
- [A Greedy Search Tree Heuristic for Symbolic Regression](https://mlsanity.com/paper/1801.01807/) (2018)
- [Learning a Formula of Interpretability to Learn Interpretable Formulas](https://mlsanity.com/paper/2004.11170/) (2020)
- [Applying Genetic Programming to Improve Interpretability in Machine Learning Models](https://mlsanity.com/paper/2005.09512/) (2020)
- [Model Learning with Personalized Interpretability Estimation (ML-PIE)](https://mlsanity.com/paper/2104.06060/) (2021)
- [Deep Learning of Part-based Representation of Data Using Sparse Autoencoders with Nonnegativity Constraints](https://mlsanity.com/paper/1601.02733/) (2016)
- [CoNSoLe: Convex Neural Symbolic Learning](https://mlsanity.com/paper/2206.00257/) (2022)
- [Solving the Exponential Growth of Symbolic Regression Trees in Geometric Semantic Genetic Programming](https://mlsanity.com/paper/1804.06808/) (2018)
- [Improving Model-based Genetic Programming for Symbolic Regression of Small Expressions](https://mlsanity.com/paper/1904.02050/) (2019)

## Built on by (most cited first, 4 of 4)

- [Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback](https://mlsanity.com/paper/2605.29184/) (2026)
- [SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression](https://mlsanity.com/paper/2401.09949/) (2024)
- [No One-Size-Fits-All Neurons: Task-based Neurons for Artificial Neural Networks](https://mlsanity.com/paper/2405.02369/) (2024)
- [GESR: A Genetic Programming-Based Symbolic Regression Method with Gene Editing](https://mlsanity.com/paper/2605.10685/) (2026)

## Abstract (arXiv)

Symbolic regression (SR) is the process of discovering hidden relationships from data with mathematical expressions, which is considered an effective way to reach interpretable machine learning (ML). Genetic programming (GP) has been the dominator in solving SR problems. However, as the scale of SR problems increases, GP often poorly demonstrates and cannot effectively address the real-world high-dimensional problems. This limitation is mainly caused by the stochastic evolutionary nature of traditional GP in constructing the trees. In this paper, we propose a differentiable approach named DGP to construct GP trees towards high-dimensional SR for the first time. Specifically, a new data structure called differentiable symbolic tree is proposed to relax the discrete structure to be continuous, thus a gradient-based optimizer can be presented for the efficient optimization. In addition, a sampling method is proposed to eliminate the discrepancy caused by the above relaxation for valid symbolic expressions. Furthermore, a diversification mechanism is introduced to promote the optimizer escaping from local optima for globally better solutions. With these designs, the proposed DGP method can efficiently search for the GP trees with higher performance, thus being capable of dealing with high-dimensional SR. To demonstrate the effectiveness of DGP, we conducted various experiments against the state of the arts based on both GP and deep neural networks. The experiment results reveal that DGP can outperform these chosen peer competitors on high-dimensional regression benchmarks with dimensions varying from tens to thousands. In addition, on the synthetic SR problems, the proposed DGP method can also achieve the best recovery rate even with different noisy levels. It is believed this work can facilitate SR being a powerful alternative to interpretable ML for a broader range of real-world problems.