# LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimization

arXiv: https://arxiv.org/abs/2602.10576 · HTML: https://mlsanity.com/paper/2602.10576/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 55 references that are in the map.
- First posted on arXiv 11 Feb 2026.

## Builds on (most cited first, 19 of 19)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery](https://mlsanity.com/paper/2405.09783/) (2024)
- [Symbolic Regression with a Learned Concept Library](https://mlsanity.com/paper/2409.09359/) (2024)
- [SymFormer: End-to-end symbolic regression using transformer-based architecture](https://mlsanity.com/paper/2205.15764/) (2022)
- [Self-Improving Language Models for Evolutionary Program Synthesis: A Case Study on ARC-AGI](https://mlsanity.com/paper/2507.14172/) (2025)
- [SR-Scientist: Scientific Equation Discovery With Agentic AI](https://mlsanity.com/paper/2510.11661/) (2025)
- [In-Context Symbolic Regression: Leveraging Large Language Models for Function Discovery](https://mlsanity.com/paper/2404.19094/) (2024)
- [Llama-3.1-FoundationAI-SecurityLLM-Base-8B Technical Report](https://mlsanity.com/paper/2504.21039/) (2025)
- [AlphaForge: A Framework to Mine and Dynamically Combine Formulaic Alpha Factors](https://mlsanity.com/paper/2406.18394/) (2024)
- [Improving exploration in policy gradient search: Application to symbolic optimization](https://mlsanity.com/paper/2107.09158/) (2021)
- [CoEvo: Continual Evolution of Symbolic Solutions Using Large Language Models](https://mlsanity.com/paper/2412.18890/) (2024)
- [DISCOVER: Deep identification of symbolically concise open-form PDEs via enhanced reinforcement-learning](https://mlsanity.com/paper/2210.02181/) (2022)

## Built on by (most cited first, 5 of 5)

- [FunctionEvolve: Structure-Guided Symbolic Regression with LLMs](https://mlsanity.com/paper/2606.07704/) (2026)
- [Large language models for partial differential equation workflows](https://mlsanity.com/paper/2608.03600/) (2026)
- [Enhancing Transformer Representations of Symbolic ODE Expressions](https://mlsanity.com/paper/2609.24746/) (2026)
- [Not Every Term Adds New Structure: Sobolev Novelty for Symbolic Regression](https://mlsanity.com/paper/2609.32597/) (2026)
- [Beyond Equation Search: Conjecture-Guided Discovery with SCOPE and ClosureBench](https://mlsanity.com/paper/2610.18938/) (2026)

## Abstract (arXiv)

Symbolic regression aims to distill mathematical equations from observational data. Recent approaches have successfully leveraged Large Language Models (LLMs) to generate equation hypotheses, capitalizing on their vast pre-trained scientific priors. However, existing frameworks predominantly treat the LLM as a static generator, relying on prompt-level guidance to steer exploration. This paradigm fails to update the model's internal representations based on search feedback, often yielding physically inconsistent or mathematically redundant expressions. In this work, we propose PiT-PO (Physics-informed Token-regularized Policy Optimization), a unified framework that evolves the LLM into an adaptive generator via reinforcement learning. Central to PiT-PO is a dual-constraint mechanism that rigorously enforces hierarchical physical validity while simultaneously applying fine-grained, token-level penalties to suppress redundant structures. Consequently, PiT-PO aligns LLM to produce equations that are both scientifically consistent and structurally parsimonious. Empirically, PiT-PO achieves state-of-the-art performance on standard benchmarks and successfully discovers novel turbulence models for challenging fluid dynamics problems. We also demonstrate that PiT-PO empowers small-scale models to outperform closed-source giants, democratizing access to high-performance scientific discovery.