# LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression

arXiv: https://arxiv.org/abs/2505.18602 · HTML: https://mlsanity.com/paper/2505.18602/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 74 references that are in the map.
- First posted on arXiv 24 May 2025.

## Builds on (most cited first, 30 of 33)

- [CodeBLEU: a Method for Automatic Evaluation of Code Synthesis](https://mlsanity.com/paper/2009.10297/) (2020)
- [ReEvo: Large Language Models as Hyper-Heuristics with Reflective Evolution](https://mlsanity.com/paper/2402.01145/) (2024)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [AutoML-Zero: Evolving Machine Learning Algorithms From Scratch](https://mlsanity.com/paper/2003.03384/) (2020)
- [Large Language Models as Evolutionary Optimizers](https://mlsanity.com/paper/2310.19046/) (2023)
- [Language Model Crossover: Variation through Few-Shot Prompting](https://mlsanity.com/paper/2302.12170/) (2023)
- [LLMOPT: Learning to Define and Solve General Optimization Problems from Scratch](https://mlsanity.com/paper/2410.13213/) (2024)
- [LLaMEA: A Large Language Model Evolutionary Algorithm for Automatically Generating Metaheuristics](https://mlsanity.com/paper/2405.20132/) (2024)
- [Symbolic Regression with a Learned Concept Library](https://mlsanity.com/paper/2409.09359/) (2024)
- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)
- [HSEvo: Elevating Automatic Heuristic Design with Diversity-Driven Harmony Search and Genetic Algorithm Using LLMs](https://mlsanity.com/paper/2412.14995/) (2024)
- [Algorithm Discovery With LLMs: Evolutionary Search Meets Reinforcement Learning](https://mlsanity.com/paper/2504.05108/) (2025)
- [Learning concise representations for regression by evolving networks of trees](https://mlsanity.com/paper/1807.00981/) (2018)
- [Evolutionary Large Language Model for Automated Feature Transformation](https://mlsanity.com/paper/2405.16203/) (2024)
- [CALM: Co-evolution of Algorithms and Language Model for Automatic Heuristic Design](https://mlsanity.com/paper/2505.12285/) (2025)
- [Shape-constrained Symbolic Regression -- Improving Extrapolation with Prior Knowledge](https://mlsanity.com/paper/2103.15624/) (2021)
- [AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors](https://mlsanity.com/paper/2406.18394/) (2024)
- [Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning](https://mlsanity.com/paper/2402.02355/) (2024)
- [Customizing Language Model Responses with Contrastive In-Context Learning](https://mlsanity.com/paper/2401.17390/) (2024)
- [Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length](https://mlsanity.com/paper/2411.03753/) (2024)
- [Automated Configuration of Genetic Algorithms by Tuning for Anytime Performance](https://mlsanity.com/paper/2106.06304/) (2021)
- [Symbolic Cognitive Diagnosis via Hybrid Optimization for Intelligent Education Systems](https://mlsanity.com/paper/2401.10840/) (2023)
- [Decision Tree Induction Through LLMs via Semantically-Aware Evolution](https://mlsanity.com/paper/2503.14217/) (2025)
- [Informed Down-Sampled Lexicase Selection: Identifying productive training cases for efficient problem solving](https://mlsanity.com/paper/2301.01488/) (2023)
- [QUBE: Enhancing Automatic Heuristic Design via Quality-Uncertainty Balanced Evolution](https://mlsanity.com/paper/2412.20694/) (2024)
- [Racing Control Variable Genetic Programming for Symbolic Regression](https://mlsanity.com/paper/2309.07934/) (2023)
- [Solving the Exponential Growth of Symbolic Regression Trees in Geometric Semantic Genetic Programming](https://mlsanity.com/paper/1804.06808/) (2018)
- [Evolving Continuous Optimisers from Scratch](https://mlsanity.com/paper/2103.11746/) (2021)

## Built on by (most cited first, 6 of 6)

- [NSR-Boost: A Neuro-Symbolic Residual Boosting Framework for Industrial Legacy Models](https://mlsanity.com/paper/2601.10457/) (2026)
- [DeepInflation: an AI agent for research and model discovery of inflation](https://mlsanity.com/paper/2601.14288/) (2026)
- [DASyR-LLM: Domain-Aware Symbolic Regression with LLMs for Kinetic Model Discovery](https://mlsanity.com/paper/2608.05120/) (2026)
- [LLMs as Post-hoc Auditors of Physiological Plausibility in Symbolic Regression: A Clinician-Evaluated Case Study](https://mlsanity.com/paper/2609.11431/) (2026)
- [EvoMO-SR: Multiobjective LLM-based Evolution of Symbolic Expressions with substructure guidance](https://mlsanity.com/paper/2609.36187/) (2026)
- [Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench](https://mlsanity.com/paper/2610.18938/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have revolutionized algorithm development, yet their application in symbolic regression, where algorithms automatically discover symbolic expressions from data, remains limited. In this paper, we propose a meta-learning framework that enables LLMs to automatically design selection operators for evolutionary symbolic regression algorithms. We first identify two key limitations in existing LLM-based algorithm evolution techniques: lack of semantic guidance and code bloat. The absence of semantic awareness can lead to ineffective exchange of useful code components, while bloat results in unnecessarily complex components; both can hinder evolutionary learning progress or reduce the interpretability of the designed algorithm. To address these issues, we enhance the LLM-based evolution framework for meta-symbolic regression with two key innovations: a complementary, semantics-aware selection operator and bloat control. Additionally, we embed domain knowledge into the prompt, enabling the LLM to generate more effective and contextually relevant selection operators. Our experimental results on symbolic regression benchmarks show that LLMs can devise selection operators that outperform nine expert-designed baselines, achieving state-of-the-art performance. Moreover, the evolved operator can further improve a state-of-the-art symbolic regression algorithm, achieving the best performance among 28 symbolic regression and other machine learning algorithms across 116 regression datasets. This demonstrates that LLMs can exceed expert-level algorithm design for symbolic regression.