# LLM4ED: Large Language Models for Automatic Equation Discovery

arXiv: https://arxiv.org/abs/2405.07761 · HTML: https://mlsanity.com/paper/2405.07761/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 57 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 29 of 29)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next](https://mlsanity.com/paper/2201.05624/) (2022)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [CodeT5+: Open Code Large Language Models for Code Understanding and Generation](https://mlsanity.com/paper/2305.07922/) (2023)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Physics-informed neural networks with hard constraints for inverse design](https://mlsanity.com/paper/2102.04626/) (2021)
- [EvoPrompt: Connecting LLMs with Evolutionary Algorithms Yields Powerful Prompt Optimizers](https://mlsanity.com/paper/2309.08532/) (2023)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Weak SINDy For Partial Differential Equations](https://mlsanity.com/paper/2007.02848/) (2020)
- [Ensemble-SINDy: Robust sparse model discovery in the low-data, high-noise limit, with active learning and control](https://mlsanity.com/paper/2111.10992/) (2021)
- [SINDy-PI: A Robust Algorithm for Parallel Implicit Sparse Identification of Nonlinear Dynamics](https://mlsanity.com/paper/2004.02322/) (2020)
- [Large Language Models as Evolutionary Optimizers](https://mlsanity.com/paper/2310.19046/) (2023)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search](https://mlsanity.com/paper/2205.13134/) (2022)
- [ODEFormer: Symbolic Regression of Dynamical Systems with Transformers](https://mlsanity.com/paper/2310.05573/) (2023)
- [SymFormer: End-to-end symbolic regression using transformer-based architecture](https://mlsanity.com/paper/2205.15764/) (2022)
- [Large Language Models As Evolution Strategies](https://mlsanity.com/paper/2402.18381/) (2024)
- [Any equation is a forest: Symbolic genetic algorithm for discovering open-form partial differential equations (SGA-PDE)](https://mlsanity.com/paper/2106.11927/) (2021)
- [Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search](https://mlsanity.com/paper/2302.11223/) (2023)
- [Probabilistic Grammars for Equation Discovery](https://mlsanity.com/paper/2012.00428/) (2020)
- [Towards Optimizing with Large Language Models](https://mlsanity.com/paper/2310.05204/) (2023)
- [SINDy-BVP: Sparse Identification of Nonlinear Dynamics for Boundary Value Problems](https://mlsanity.com/paper/2005.10756/) (2020)
- [ChatGPT and Other Large Language Models as Evolutionary Engines for Online Interactive Collaborative Game Design](https://mlsanity.com/paper/2303.02155/) (2023)
- [Integration of knowledge and data in machine learning](https://mlsanity.com/paper/2202.10337/) (2022)
- [Deep learning algorithm for data-driven simulation of noisy dynamical system](https://mlsanity.com/paper/1802.08323/) (2018)
- [Physics-constrained robust learning of open-form partial differential equations from limited and noisy data](https://mlsanity.com/paper/2309.07672/) (2023)

## Built on by (most cited first, 14 of 14)

- [A Systematic Survey on Large Language Models for Algorithm Design](https://mlsanity.com/paper/2410.14716/) (2024)
- [LLM-based Agentic Reasoning Frameworks: A Survey from Methods to Scenarios](https://mlsanity.com/paper/2508.17692/) (2025)
- [PhysGym: Benchmarking LLMs in Interactive Physics Discovery with Controlled Priors](https://mlsanity.com/paper/2507.15550/) (2025)
- [Generative Discovery of Partial Differential Equations by Learning from Math Handbooks](https://mlsanity.com/paper/2505.05869/) (2025)
- [Position: The ML Community Must Build an AI-Augmented Peer-Review Ecosystem](https://mlsanity.com/paper/2506.08134/) (2025)
- [Analytical Survey of Learning with Low-Resource Data: From Analysis to Investigation](https://mlsanity.com/paper/2510.08962/) (2025)
- [OpInf-LLM: Parametric PDE Solving with LLMs via Operator Inference](https://mlsanity.com/paper/2602.01493/) (2026)
- [A Data-Driven Framework for Discovering Fractional Differential Equations in Complex Systems](https://mlsanity.com/paper/2412.03970/) (2024)
- [STRIDE: A Self-Reflective Agent Framework for Reliable Automatic Equation Discovery](https://mlsanity.com/paper/2605.17790/) (2026)
- [Dynamic Bayesian Optimization Framework for Instruction Tuning in Partial Differential Equation Discovery](https://mlsanity.com/paper/2601.00088/) (2025)
- [Joint discovery of governing partial differential equations from multi-source datasets by competitive optimization](https://mlsanity.com/paper/2606.30699/) (2026)
- [LLM-Guided ODE Discovery and Parameter Inference from Small-Cohort Aggregate Data](https://mlsanity.com/paper/2607.00733/) (2026)
- [Large language models for partial differential equation workflows](https://mlsanity.com/paper/2608.03600/) (2026)
- [DASyR-LLM: Domain-Aware Symbolic Regression with LLMs for Kinetic Model Discovery](https://mlsanity.com/paper/2608.05120/) (2026)

## Abstract (arXiv)

Equation discovery is aimed at directly extracting physical laws from data and has emerged as a pivotal research domain. Previous methods based on symbolic mathematics have achieved substantial advancements, but often require the design of implementation of complex algorithms. In this paper, we introduce a new framework that utilizes natural language-based prompts to guide large language models (LLMs) in automatically mining governing equations from data. Specifically, we first utilize the generation capability of LLMs to generate diverse equations in string form, and then evaluate the generated equations based on observations. In the optimization phase, we propose two alternately iterated strategies to optimize generated equations collaboratively. The first strategy is to take LLMs as a black-box optimizer and achieve equation self-improvement based on historical samples and their performance. The second strategy is to instruct LLMs to perform evolutionary operators for global search. Experiments are extensively conducted on both partial differential equations and ordinary differential equations. Results demonstrate that our framework can discover effective equations to reveal the underlying physical laws under various nonlinear dynamic systems. Further comparisons are made with state-of-the-art models, demonstrating good stability and usability. Our framework substantially lowers the barriers to learning and applying equation discovery techniques, demonstrating the application potential of LLMs in the field of knowledge discovery.