# Efficient Constant Optimization for Symbolic Regression with GPU-Accelerated Tree-Based Genetic Programming

arXiv: https://arxiv.org/abs/2609.03352 · HTML: https://mlsanity.com/paper/2609.03352/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 3 Sep 2026.

## Builds on (most cited first, 15 of 15)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [DLGA-PDE: Discovery of PDEs with incomplete candidate library via combination of deep learning and genetic algorithm](https://mlsanity.com/paper/2001.07305/) (2020)
- [Where are we now? A large benchmark study of recent symbolic regression methods](https://mlsanity.com/paper/1804.09331/) (2018)
- [Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)](https://mlsanity.com/paper/2106.11927/) (2021)
- [TensorGP -- Genetic Programming Engine in TensorFlow](https://mlsanity.com/paper/2103.07512/) (2021)
- [Enabling Population-Level Parallelism in Tree-Based Genetic Programming for GPU Acceleration](https://mlsanity.com/paper/2501.17168/) (2025)
- [Kozax: Flexible and Scalable Genetic Programming in JAX](https://mlsanity.com/paper/2502.03047/) (2025)
- [Im2win: An Efficient Convolution Paradigm on GPU](https://mlsanity.com/paper/2306.14316/) (2023)
- [Data-driven discovery of governing differential equations across physical systems](https://mlsanity.com/paper/2606.09638/) (2026)
- [A note on solving nonlinear optimization problems in variable precision](https://mlsanity.com/paper/1812.03467/) (2018)
- [Speed Benchmarking of Genetic Programming Frameworks](https://mlsanity.com/paper/2106.11919/) (2021)
- [Local Optimization Often is Ill-conditioned in Genetic Programming for Symbolic Regression](https://mlsanity.com/paper/2209.00942/) (2022)
- [TROPHY: Trust Region Optimization Using a Precision Hierarchy](https://mlsanity.com/paper/2202.08387/) (2022)
- [Enabling Memory-efficient Im2win Convolution with Multi-precision Support on GPU CUDA and Tensor Cores](https://mlsanity.com/paper/2608.20725/) (2026)

## Built on by (most cited first, 1 of 1)

- [MorphingNet: Symbolic Regression Where the Network Is the Formula](https://mlsanity.com/paper/2610.19942/) (2026)

## Abstract (arXiv)

Constant optimization refines the numerical coefficients of candidate expressions in tree-based genetic programming for symbolic regression. But its per-generation cost has led modern GPU-accelerated frameworks to omit it or restrict it to lightweight forms. We present a GPU-resident, batched Levenberg--Marquardt solver that optimizes constants across a structurally heterogeneous population of expression trees using a fixed number of population-wide CUDA launches per iteration. Reverse-mode automatic differentiation assembles the per-tree Jacobian in one backward sweep, making the dominant per-iteration cost independent of the number of constants per tree, and a double-precision delivery guard guarantees that returned constants are never worse than their initial values. On early-generation populations, the solver sustains up to 5.1×105 trees per second on an NVIDIA A100; at a GPU-saturated benchmark configuration it delivers roughly 9.9× the throughput of Operon running on a 64-core EPYC 7763, while matching fp64-reference quality. Integrated in-process into EvoGP, the solver enables end-to-end search to recover governing equations on 10 of 18 constructed problems versus 0 for stock EvoGP. Our code is at https://github.com/TensorConv/CuSR.