# Not Every Term Adds New Structure: Sobolev Novelty for Symbolic Regression

arXiv: https://arxiv.org/abs/2609.32597 · HTML: https://mlsanity.com/paper/2609.32597/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 34 references that are in the map.
- First posted on arXiv 26 Sep 2026.

## Builds on (most cited first, 22 of 22)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [Sobolev Training for Neural Networks](https://mlsanity.com/paper/1706.04859/) (2017)
- [LLM-SR: Scientific Equation Discovery via Programming with Large Language Models](https://mlsanity.com/paper/2404.18400/) (2024)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [Symbolic Physics Learner: Discovering governing equations via Monte Carlo tree search](https://mlsanity.com/paper/2205.13134/) (2022)
- [SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training](https://mlsanity.com/paper/2310.02227/) (2023)
- [Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search](https://mlsanity.com/paper/2302.11223/) (2023)
- [Learning concise representations for regression by evolving networks of trees](https://mlsanity.com/paper/1807.00981/) (2018)
- [Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length](https://mlsanity.com/paper/2411.03753/) (2024)
- [Call for Action: towards the next generation of symbolic regression benchmark](https://mlsanity.com/paper/2505.03977/) (2025)
- [A Neural Symbolic Model for Space Physics](https://mlsanity.com/paper/2503.07994/) (2025)
- [Influence-Guided Symbolic Regression: Scientific Discovery via LLM-Driven Equation Search with Granular Feedback](https://mlsanity.com/paper/2605.29184/) (2026)
- [RSRM: Reinforcement Symbolic Regression Machine](https://mlsanity.com/paper/2305.14656/) (2023)
- [LLM-Based Scientific Equation Discovery via Physics-Informed Token-Regularized Policy Optimization](https://mlsanity.com/paper/2602.10576/) (2026)
- [Beyond Accuracy and Complexity: The Effective Information Criterion for Structurally Stable Symbolic Regression](https://mlsanity.com/paper/2509.21780/) (2025)

## Abstract (arXiv)

Symbolic regression (SR) aims to discover compact and meaningful mathematical equations from data, but searching the vast combinatorial space of symbolic structures remains challenging. Existing methods typically guide this process using expression-level objectives, such as fitting error, which assess a candidate equation as a whole but provide little information about whether an individual term contributes genuinely new structure or is largely redundant with the rest of the expression. We introduce Sobolev Novelty, a term-level measure of structural independence for symbolic equations. For each term, we construct an empirical Sobolev signature from its function values and exact derivatives over the observed inputs, and quantify how much of this behavior cannot be reconstructed by the remaining terms. We further derive a theory-calibrated threshold, yielding a principled and tuning-free criterion for identifying structurally novel terms. Using this threshold, 92.6% of terms in benchmark ground-truth equations exhibit sufficient structural novelty, compared with only 38.3% on average for expressions produced by 15 SR methods, revealing a substantial gap between scientific equations and current SR solutions. As a lightweight plug-in, Sobolev Novelty can be incorporated into diverse SR paradigms to support term pruning, search guidance, LLM feedback, and data selection, yielding consistent performance gains and demonstrating broad applicability.