# Symbolic Regression with a Learned Concept Library

arXiv: https://arxiv.org/abs/2409.09359 · HTML: https://mlsanity.com/paper/2409.09359/

## Key facts

- Cited by 83 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 56 references that are in the map.
- First posted on arXiv 14 Sep 2024.

## Builds on (most cited first, 30 of 31)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Training Compute-Optimal Large Language Models](https://mlsanity.com/paper/2203.15556/) (2022)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters](https://mlsanity.com/paper/2408.03314/) (2024)
- [Competition-Level Code Generation with AlphaCode](https://mlsanity.com/paper/2203.07814/) (2022)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [ViperGPT: Visual Inference via Python Execution for Reasoning](https://mlsanity.com/paper/2303.08128/) (2023)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Visual Programming: Compositional visual reasoning without training](https://mlsanity.com/paper/2211.11559/) (2022)
- [Discovering Symbolic Models from Deep Learning with Inductive Biases](https://mlsanity.com/paper/2006.11287/) (2020)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [DreamCoder: Growing generalizable, interpretable knowledge with wake-sleep Bayesian program learning](https://mlsanity.com/paper/2006.08381/) (2020)
- [LLM-SR: Scientific Equation Discovery via Programming with Large Language Models](https://mlsanity.com/paper/2404.18400/) (2024)
- [Revisiting Neural Scaling Laws in Language and Vision](https://mlsanity.com/paper/2209.06640/) (2022)
- [RobustFill: Neural Program Learning under Noisy I/O](https://mlsanity.com/paper/1703.07469/) (2017)
- [Language Model Crossover: Variation through Few-Shot Prompting](https://mlsanity.com/paper/2302.12170/) (2023)
- [Broken Neural Scaling Laws](https://mlsanity.com/paper/2210.14891/) (2022)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [Parsel: Algorithmic Reasoning with Language Models by Composing Decompositions](https://mlsanity.com/paper/2212.10561/) (2022)
- [Top-Down Synthesis for Library Learning](https://mlsanity.com/paper/2211.16605/) (2022)
- [LILO: Learning Interpretable Libraries by Compressing and Documenting Code](https://mlsanity.com/paper/2310.19791/) (2023)
- [Leveraging Language to Learn Program Abstractions and Search Heuristics](https://mlsanity.com/paper/2106.11053/) (2021)
- [SpreadsheetCoder: Formula Prediction from Semi-structured Context](https://mlsanity.com/paper/2106.15339/) (2021)
- [Program Synthesis and Semantic Parsing with Learned Code Idioms](https://mlsanity.com/paper/1906.10816/) (2019)
- [Learning Differentiable Programs with Admissible Neural Heuristics](https://mlsanity.com/paper/2007.12101/) (2020)
- [Fast, accurate, and transferable many-body interatomic potentials by symbolic regression](https://mlsanity.com/paper/1904.01095/) (2019)

## Built on by (most cited first, 30 of 83)

- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://mlsanity.com/paper/2506.13131/) (2025)
- [LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Models](https://mlsanity.com/paper/2504.10415/) (2025)
- [SR-Scientist: Scientific Equation Discovery With Agentic AI](https://mlsanity.com/paper/2510.11661/) (2025)
- [DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience](https://mlsanity.com/paper/2506.04282/) (2025)
- [Think like a Scientist: Physics-guided LLM Agent for Equation Discovery](https://mlsanity.com/paper/2602.12259/) (2026)
- [Autonomous Code Evolution Meets NP-Completeness](https://mlsanity.com/paper/2509.07367/) (2025)
- [AlphaOPT: Formulating Optimization Programs with Self-Improving LLM Experience Library](https://mlsanity.com/paper/2510.18428/) (2025)
- [LLM-AutoSciLab: Closed-Loop Scientific Discovery via Active Experimentation with LLMs](https://mlsanity.com/paper/2605.24043/) (2026)
- [A Neural Symbolic Model for Space Physics](https://mlsanity.com/paper/2503.07994/) (2025)
- [Can We Predict Before Executing Machine Learning Agents?](https://mlsanity.com/paper/2601.05930/) (2026)
- [GENSR: Symbolic Regression Based in Equation Generative Space](https://mlsanity.com/paper/2602.20557/) (2026)
- [Autonomous Research Agents: A Survey of AI Scientists and the Verification Gap](https://mlsanity.com/paper/2608.05179/) (2026)
- [In Context Learning and Reasoning for Symbolic Regression with Large Language Models](https://mlsanity.com/paper/2410.17448/) (2024)
- [Mimicking the Physicist's Eye:A VLM-centric Approach for Physics Formula Discovery](https://mlsanity.com/paper/2508.17380/) (2025)
- [LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Search](https://mlsanity.com/paper/2606.25039/) (2026)
- [Discovering Symbolic Differential Equations with Symmetry Invariants](https://mlsanity.com/paper/2505.12083/) (2025)
- [LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression](https://mlsanity.com/paper/2505.18602/) (2025)
- [Man-Made Heuristics Are Dead. Long Live Code Generators!](https://mlsanity.com/paper/2510.08803/) (2025)
- [Foundation Models for Scientific Discovery: From Paradigm Enhancement to Paradigm Transition](https://mlsanity.com/paper/2510.15280/) (2025)
- [When is a System Discoverable from Data? Discovery Requires Chaos](https://mlsanity.com/paper/2511.08860/) (2025)
- [A-SR: Self-Evolving Agentic LLMs for Symbolic Regression via Hierarchical Coordination](https://mlsanity.com/paper/2608.04872/) (2026)
- [OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inference](https://mlsanity.com/paper/2602.01493/) (2026)
- [LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimization](https://mlsanity.com/paper/2602.10576/) (2026)
- [What Makes an LLM a Good Optimizer? A Trajectory Analysis of LLM-Guided Evolutionary Search](https://mlsanity.com/paper/2604.19440/) (2026)
- [Generative Design of Functional Metal Complexes Utilizing the Internal Knowledge of Large Language Models](https://mlsanity.com/paper/2410.18136/) (2024)
- [Large Language Models for Extrapolative Modeling of Manufacturing Processes](https://mlsanity.com/paper/2502.12185/) (2025)
- [Physics-Grounded Motion Forecasting via Equation Discovery for Trajectory-Guided Image-to-Video Generation](https://mlsanity.com/paper/2507.06830/) (2025)
- [Program Synthesis via Test-Time Transduction](https://mlsanity.com/paper/2509.17393/) (2025)
- [DiSciPLE: Learning Interpretable Programs for Scientific Visual Discovery](https://mlsanity.com/paper/2502.10060/) (2025)
- [Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networks](https://mlsanity.com/paper/2505.07956/) (2025)

## Abstract (arXiv)

We present a novel method for symbolic regression (SR), the task of searching for compact programmatic hypotheses that best explain a dataset. The problem is commonly solved using genetic algorithms; we show that we can enhance such methods by inducing a library of abstract textual concepts. Our algorithm, called LaSR, uses zero-shot queries to a large language model (LLM) to discover and evolve concepts occurring in known high-performing hypotheses. We discover new hypotheses using a mix of standard evolutionary steps and LLM-guided steps (obtained through zero-shot LLM queries) conditioned on discovered concepts. Once discovered, hypotheses are used in a new round of concept abstraction and evolution. We validate LaSR on the Feynman equations, a popular SR benchmark, as well as a set of synthetic tasks. On these benchmarks, LaSR substantially outperforms a variety of state-of-the-art SR approaches based on deep learning and evolutionary algorithms. Moreover, we show that LaSR can be used to discover a novel and powerful scaling law for LLMs.