# Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery

arXiv: https://arxiv.org/abs/2206.10540 · HTML: https://mlsanity.com/paper/2206.10540/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 68 references that are in the map.
- First posted on arXiv 21 Jun 2022.

## Builds on (most cited first, 25 of 25)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [AI Poincar'e: Machine Learning Conservation Laws from Trajectories](https://mlsanity.com/paper/2011.04698/) (2020)
- [Where are we now? A large benchmark study of recent symbolic regression methods](https://mlsanity.com/paper/1804.09331/) (2018)
- [Learning concise representations for regression by evolving networks of trees](https://mlsanity.com/paper/1807.00981/) (2018)
- [torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation](https://mlsanity.com/paper/2011.12913/) (2020)
- [Symbolic regression for scientific discovery: an application to wind speed forecasting](https://mlsanity.com/paper/2102.10570/) (2021)
- [Scheduled Sampling Based on Decoding Steps for Neural Machine Translation](https://mlsanity.com/paper/2108.12963/) (2021)
- [Machine-Learning Non-Conservative Dynamics for New-Physics Detection](https://mlsanity.com/paper/2106.00026/) (2021)

## Built on by (most cited first, 30 of 46)

- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)
- [Symbolic Regression is NP-hard](https://mlsanity.com/paper/2207.01018/) (2022)
- [LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models](https://mlsanity.com/paper/2504.10415/) (2025)
- [NewtonBench: Benchmarking Generalizable Scientific Law Discovery in LLM Agents](https://mlsanity.com/paper/2510.07172/) (2025)
- [Deep Learning Symmetries and Their Lie Groups, Algebras, and Subalgebras from First Principles](https://mlsanity.com/paper/2301.05638/) (2023)
- [Call for Action: towards the next generation of symbolic regression benchmark](https://mlsanity.com/paper/2505.03977/) (2025)
- [PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors](https://mlsanity.com/paper/2507.15550/) (2025)
- [ParFam -- (Neural Guided) Symbolic Regression Based on Continuous Global Optimization](https://mlsanity.com/paper/2310.05537/) (2023)
- [Symbolic Regression via Control Variable Genetic Programming](https://mlsanity.com/paper/2306.08057/) (2023)
- [A Transformer Model for Symbolic Regression towards Scientific Discovery](https://mlsanity.com/paper/2312.04070/) (2023)
- [Is the Machine Smarter than the Theorist: Deriving Formulas for Particle Kinematics with Symbolic Regression](https://mlsanity.com/paper/2211.08420/) (2022)
- [Mimicking the Physicist's Eye:A VLM-centric Approach for Physics Formula Discovery](https://mlsanity.com/paper/2508.17380/) (2025)
- [Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Models](https://mlsanity.com/paper/2509.03036/) (2025)
- [Autonomous Data Agents: A New Opportunity for Smart Data](https://mlsanity.com/paper/2509.18710/) (2025)
- [A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination](https://mlsanity.com/paper/2608.04872/) (2026)
- [Oracle-Preserving Latent Flows](https://mlsanity.com/paper/2302.00806/) (2023)
- [Exploring the Truth and Beauty of Theory Landscapes with Machine Learning](https://mlsanity.com/paper/2401.11513/) (2024)
- [TorchSISSO: A PyTorch-Based Implementation of the Sure Independence Screening and Sparsifying Operator for Efficient and Interpretable Model Discovery](https://mlsanity.com/paper/2410.01752/) (2024)
- [Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large p](https://mlsanity.com/paper/2410.13681/) (2024)
- [Analyzing Generalization in Pre-Trained Symbolic Regression](https://mlsanity.com/paper/2509.19849/) (2025)
- [Bayesian Symbolic Regression via Posterior Sampling](https://mlsanity.com/paper/2512.10849/) (2025)
- [Seeking Truth and Beauty in Flavor Physics with Machine Learning](https://mlsanity.com/paper/2311.00087/) (2023)
- [An Efficient and Generalizable Symbolic Regression Method for Time Series Analysis](https://mlsanity.com/paper/2409.03986/) (2024)
- [Sheaf-Theoretic Transport and Obstruction for Detecting Scientific Theory Shift in AI Agents](https://mlsanity.com/paper/2605.14033/) (2026)
- [STRIDE: A Self-Reflective Agent Framework for Reliable Automatic Equation Discovery](https://mlsanity.com/paper/2605.17790/) (2026)
- [Vertical Symbolic Regression](https://mlsanity.com/paper/2312.11955/) (2023)
- [Symbolic Quantile Regression for the Interpretable Prediction of Conditional Quantiles](https://mlsanity.com/paper/2508.08080/) (2025)
- [Beyond Benchmarks: LLM Evaluation with an Anthropomorphic and Lifecycle-oriented Roadmap](https://mlsanity.com/paper/2508.18646/) (2025)
- [SURFACEBENCH: A Geometry-Aware Benchmark for Symbolic Surface Discovery](https://mlsanity.com/paper/2511.10833/) (2025)
- [Analyzing Error Sources in Global Feature Effect Estimation](https://mlsanity.com/paper/2603.15057/) (2026)

## Abstract (arXiv)

This paper revisits datasets and evaluation criteria for Symbolic Regression (SR), specifically focused on its potential for scientific discovery. Focused on a set of formulas used in the existing datasets based on Feynman Lectures on Physics, we recreate 120 datasets to discuss the performance of symbolic regression for scientific discovery (SRSD). For each of the 120 SRSD datasets, we carefully review the properties of the formula and its variables to design reasonably realistic sampling ranges of values so that our new SRSD datasets can be used for evaluating the potential of SRSD such as whether or not an SR method can (re)discover physical laws from such datasets. We also create another 120 datasets that contain dummy variables to examine whether SR methods can choose necessary variables only. Besides, we propose to use normalized edit distances (NED) between a predicted equation and the true equation trees for addressing a critical issue that existing SR metrics are either binary or errors between the target values and an SR model's predicted values for a given input. We conduct benchmark experiments on our new SRSD datasets using various representative SR methods. The experimental results show that we provide a more realistic performance evaluation, and our user study shows that the NED correlates with human judges significantly more than an existing SR metric. We publish repositories of our code and 240 SRSD datasets.