# Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length

arXiv: https://arxiv.org/abs/2411.03753 · HTML: https://mlsanity.com/paper/2411.03753/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 69 references that are in the map.
- First posted on arXiv 6 Nov 2024.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [PMLB: A Large Benchmark Suite for Machine Learning Evaluation and Comparison](https://mlsanity.com/paper/1703.00512/) (2017)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search](https://mlsanity.com/paper/2205.13134/) (2022)
- [Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws](https://mlsanity.com/paper/2303.03192/) (2023)
- [Deep Symbolic Regression for Recurrent Sequences](https://mlsanity.com/paper/2201.04600/) (2022)
- [A-NeSI: A Scalable Approximate Method for Probabilistic Neurosymbolic Inference](https://mlsanity.com/paper/2212.12393/) (2022)
- [SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training](https://mlsanity.com/paper/2310.02227/) (2023)
- [Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search](https://mlsanity.com/paper/2302.11223/) (2023)
- [Learning concise representations for regression by evolving networks of trees](https://mlsanity.com/paper/1807.00981/) (2018)
- [Epsilon-Lexicase Selection for Regression](https://mlsanity.com/paper/1905.13266/) (2019)
- [Attentive Pooling Networks](https://mlsanity.com/paper/1602.03609/) (2016)
- [Controllable Neural Symbolic Regression](https://mlsanity.com/paper/2304.10336/) (2023)
- [Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression](https://mlsanity.com/paper/2302.00773/) (2023)
- [RSRM: Reinforcement Symbolic Regression Machine](https://mlsanity.com/paper/2305.14656/) (2023)
- [Automated discovery of symbolic laws governing skill acquisition from naturally occurring data](https://mlsanity.com/paper/2404.05689/) (2024)
- [A Comprehensive Survey on Artificial Intelligence for Complex Network: Potential, Methodology and Application](https://mlsanity.com/paper/2402.16887/) (2024)

## Built on by (most cited first, 17 of 17)

- [DrSR: LLM based Scientific Equation Discovery with Dual Reasoning from Data and Experience](https://mlsanity.com/paper/2506.04282/) (2025)
- [GENSR: Symbolic Regression Based in Equation Generative Space](https://mlsanity.com/paper/2602.20557/) (2026)
- [LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression](https://mlsanity.com/paper/2505.18602/) (2025)
- [From Observation to Insight: Mechanistic World Models and the Quest for Autonomous Discovery](https://mlsanity.com/paper/2607.12474/) (2026)
- [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025)
- [Breaking the Simplification Bottleneck in Amortized Neural Symbolic Regression](https://mlsanity.com/paper/2602.08885/) (2026)
- [Beyond Reinforcement Learning: Fast and Scalable Quantum Circuit Synthesis](https://mlsanity.com/paper/2602.15146/) (2026)
- [Beyond Accuracy and Complexity: The Effective Information Criterion for Structurally Stable Symbolic Regression](https://mlsanity.com/paper/2509.21780/) (2025)
- [Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization](https://mlsanity.com/paper/2604.08324/) (2026)
- [Discovering Ordinary Differential Equations with LLM-Based Qualitative and Quantitative Evaluation](https://mlsanity.com/paper/2605.07323/) (2026)
- [EditSR: Enhancing Neural Symbolic Regression via Edit-based Rectification](https://mlsanity.com/paper/2606.07915/) (2026)
- [On the post-hoc Evaluation of PDE Discovery: A Multifaceted Challenge of Scientific Advancement](https://mlsanity.com/paper/2607.23753/) (2026)
- [AutoSR: Automatic Symbolic Regression by Searching Research States](https://mlsanity.com/paper/2608.16876/) (2026)
- [MOSAIC-SR: Transformer-Guided Symbolic Regression for Scientific Equation Recovery](https://mlsanity.com/paper/2609.20997/) (2026)
- [SNIP++: Fine-Grained Symbolic-Numerical Alignment for Symbolic Regression](https://mlsanity.com/paper/2609.31965/) (2026)
- [Not Every Term Adds New Structure: Sobolev Novelty for Symbolic Regression](https://mlsanity.com/paper/2609.32597/) (2026)
- [DASS: A Solver-Adaptable Dynamic Auxiliary Search Strategy for Symbolic Regression](https://mlsanity.com/paper/2610.16812/) (2026)

## Abstract (arXiv)

Symbolic regression, a task discovering the formula best fitting the given data, is typically based on the heuristical search. These methods usually update candidate formulas to obtain new ones with lower prediction errors iteratively. However, since formulas with similar function shapes may have completely different symbolic forms, the prediction error does not decrease monotonously as the search approaches the target formula, causing the low recovery rate of existing methods. To solve this problem, we propose a novel search objective based on the minimum description length, which reflects the distance from the target and decreases monotonically as the search approaches the correct form of the target formula. To estimate the minimum description length of any input data, we design a neural network, MDLformer, which enables robust and scalable estimation through large-scale training. With the MDLformer's output as the search objective, we implement a symbolic regression method, SR4MDL, that can effectively recover the correct mathematical form of the formula. Extensive experiments illustrate its excellent performance in recovering formulas from data. Our method successfully recovers around 50 formulas across two benchmark datasets comprising 133 problems, outperforming state-of-the-art methods by 43.92%. Experiments on 122 unseen black-box problems further demonstrate its generalization performance. We release our code at https://github.com/tsinghua-fib-lab/SR4MDL .