mlsanityData to 14 Oct 2026

In-Context Symbolic Regression: Leveraging Large Language Models for Function Discovery

arXiv 2404.19094 ↗First posted 29 Apr 2024cs.CL cs.LGcited by 31

Not on the map. Why?

Abstract (arXiv)

State of the art Symbolic Regression (SR) methods currently build specialized models, while the application of Large Language Models (LLMs) remains largely unexplored. In this work, we introduce the first comprehensive framework that utilizes LLMs for the task of SR. We propose In-Context Symbolic Regression (ICSR), an SR method which iteratively refines a functional form with an LLM and determines its coefficients with an external optimizer. ICSR leverages LLMs' strong mathematical prior both to propose an initial set of possible functions given the observations and to refine them based on their errors. Our findings reveal that LLMs are able to successfully find symbolic equations that fit the given data, matching or outperforming the overall performance of the best SR baselines on four popular benchmarks, while yielding simpler equations with better out of distribution generalization.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 29 Apr 2024.

Builds on

28 of 48 references.

References · 28

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  10. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  12. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
16 more
  1. 2023Lost in the Middle: How Language Models Use Long Contextscited by 3,455
  2. 2023Vision-Language Models for Vision Tasks: A Surveycited by 839
  3. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  4. 2023Otter: A Multi-Modal Model with In-Context Instruction Tuningcited by 550
  5. 2019AI Feynman: a Physics-Inspired Method for Symbolic Regressioncited by 548
  6. 2023Large Language Models Are Zero-Shot Time Series Forecasterscited by 500
  7. 2019Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradientscited by 309
  8. 2019Deep Learning for Symbolic Mathematicscited by 270
  9. 2023Generating Images with Multimodal Language Modelscited by 260
  10. 2022End-to-end symbolic regression with transformerscited by 203
  11. 2023Large Language Models as General Pattern Machinescited by 197
  12. 2021Neural Symbolic Regression that Scalescited by 192
  13. 2021SymbolicGPT: A Generative Transformer Model for Symbolic Regressioncited by 105
  14. 2023Transformer-based Planning for Symbolic Regressioncited by 79
  15. 2023OtterHD: A High-Resolution Multi-modality Modelcited by 72
  16. 2023Transformers Learn to Achieve Second-Order Convergence Rates for In-Context Linear Regressioncited by 49

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 18; 2026: 13.

31 papers.

Sort:

By area:

  1. 2025LLM-SRBench: A New Benchmark for Scientific Equation Discovery with Large Language Modelscited by 71
  2. 2025DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experiencecited by 22
  3. 2026Think like a Scientist: Physics-guided LLM Agent for Equation Discoverycited by 16
  4. 2025PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priorscited by 15
  5. 2025Knowledge Integration for Physics-informed Symbolic Regression Using Pre-trained Large Language Modelscited by 7
  6. 2025MDBench: Benchmarking Data-Driven Methods for Model Discoverycited by 7
  7. 2026LLM-ACES: Closed-Loop Discovery of Dynamical Systems with LLM-Guided Adaptive Searchcited by 7
  8. 2025Discovering Symbolic Differential Equations with Symmetry Invariantscited by 6
  9. 2025Can Large Language Models Adequately Perform Symbolic Reasoning Over Time Series?cited by 6
  10. 2026LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimizationcited by 5
  11. 2025Large Language Models for Extrapolative Modeling of Manufacturing Processescited by 4
  12. 2026LLM-ODE: Data-driven Discovery of Dynamical Systems with Large Language Modelscited by 4
19 more
  1. 2025DiSciPLE: Learning Interpretable Programs for Scientific Visual Discoverycited by 3
  2. 2025Symbolic Regression with Multimodal Large Language Models and Kolmogorov Arnold Networkscited by 3
  3. 2025Analyzing Generalization in Pre-Trained Symbolic Regressioncited by 3
  4. 2025Automated Model Discovery via Multi-modal & Multi-step Pipelinecited by 3
  5. 2025Data-driven Discovery of Digital Twins in Biomedical Researchcited by 2
  6. 2025EGG-SR: Embedding Symbolic Equivalence into Symbolic Regression via Equality Graphcited by 2
  7. 2026STRIDE: A Self-Reflective Agent Framework for Reliable Automatic Equation Discoverycited by 2
  8. 2025Parsing the Language of Expression: Enhancing Symbolic Regression with Domain-Aware Symbolic Priorscited by 1
  9. 2025Can Test-time Computation Mitigate Reproduction Bias in Neural Symbolic Regression?cited by 1
  10. 2025DeepRule: An Integrated Framework for Automated Business Rule Generation via Deep Predictive Modeling and Hybrid Search Optimizationcited by 1
  11. 2025Dynamic Bayesian Optimization Framework for Instruction Tuning in Partial Differential Equation Discoverycited by 1
  12. 2026Discovering Ordinary Differential Equations with LLM-Based Qualitative and Quantitative Evaluationcited by 0
  13. 2026ERBench: A Benchmark and Testsuite for Equation Discovery Algorithmscited by 0
  14. 2026Decision-Making under Combinatorial Riskcited by 0
  15. 2026SciText2Eq: Assessing LLMs for Explainable Equation Generation for Scientific Creativitycited by 0
  16. 2026Verbalized Particle Posterior: Bayesian Inference over Natural Language Hypothesescited by 0
  17. 2026AutoSR: Automatic Symbolic Regression by Searching Research Statescited by 0
  18. 2026Coupled Harness--Model Evolution for Scientific Equation Discoverycited by 0
  19. 2026Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBenchcited by 0