# When Good Equations Get Bad Scores: Improving Symbolic Regression Through Better Parameter Optimization

arXiv: https://arxiv.org/abs/2605.23272 · HTML: https://mlsanity.com/paper/2605.23272/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 41 references that are in the map.
- First posted on arXiv 22 May 2026.

## Builds on (most cited first, 12 of 12)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [Symbolic Regression with a Learned Concept Library](https://mlsanity.com/paper/2409.09359/) (2024)
- [Benchmarking symbolic regression constant optimization schemes](https://mlsanity.com/paper/2412.02126/) (2024)
- [Improving Monte Carlo Tree Search for Symbolic Regression](https://mlsanity.com/paper/2509.15929/) (2025)
- [Deep Symbolic Optimization: Reinforcement Learning for Symbolic Mathematics](https://mlsanity.com/paper/2505.10762/) (2025)
- [Symbolic Regression with Fast Function Extraction and Nonlinear Least Squares Optimization](https://mlsanity.com/paper/2209.09675/) (2022)
- [Finetuning Large Language Model as an Effective Symbolic Regressor](https://mlsanity.com/paper/2508.09897/) (2025)
- [An Empirical Investigation of Neural ODEs and Symbolic Regression for Dynamical Systems](https://mlsanity.com/paper/2601.20637/) (2026)

## Abstract (arXiv)

Symbolic Regression (SR) plays a central role in scientific knowledge discovery by distilling mathematical equations from observational data. Most existing SR methods function within a bi-level optimization framework: an outer loop that searches for the discrete equation structure, and an inner loop that optimizes the continuous parameters of that structure. Crucially, parameter-fitting quality directly determines a structure's score and thus the outer-loop search. However, nonlinear operators make the inner loop highly non-convex, and budget-driven reliance on fast local solvers (e.g., BFGS) often yields poor local minima and underestimated scores for correct structures. This ``Good Structure, Bad Score'' phenomenon becomes a key bottleneck, degrading efficiency and misguiding the search away from the true equation. To resolve this, we propose SAGE-Fit (Structure-Aware and Semantics-Guided Evaluator for Symbolic Regression), an SR-native fitting framework that exploits the dual native priors of symbolic expressions. By capitalizing on the structural and semantic priors unique to SR, we design tailored modules for each property, thereby effectively mitigating this optimization bottleneck. Extensive experiments demonstrate that our approach, as a plug-and-play module, significantly enhances evaluation fidelity and universally improves the performance of various SR systems.