# Shape-constrained Symbolic Regression -- Improving Extrapolation with Prior Knowledge

arXiv: https://arxiv.org/abs/2103.15624 · HTML: https://mlsanity.com/paper/2103.15624/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 59 references that are in the map.
- First posted on arXiv 29 Mar 2021.

## Builds on (most cited first, 15 of 15)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Physics-Constrained Deep Learning for High-dimensional Surrogate Modeling and Uncertainty Quantification without Labeled Data](https://mlsanity.com/paper/1901.06314/) (2019)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [Label-Free Supervision of Neural Networks with Physics and Domain Knowledge](https://mlsanity.com/paper/1609.05566/) (2016)
- [Where are we now? A large benchmark study of recent symbolic regression methods](https://mlsanity.com/paper/1804.09331/) (2018)
- [Monotonic Calibrated Interpolated Look-Up Tables](https://mlsanity.com/paper/1505.06378/) (2015)
- [DSOS and SDSOS Optimization: More Tractable Alternatives to Sum of Squares and Semidefinite Optimization](https://mlsanity.com/paper/1706.02586/) (2017)
- [A Greedy Search Tree Heuristic for Symbolic Regression](https://mlsanity.com/paper/1801.01807/) (2018)
- [Nonparametric Shape-restricted Regression](https://mlsanity.com/paper/1709.05707/) (2017)
- [Neural-Guided Symbolic Regression with Asymptotic Constraints](https://mlsanity.com/paper/1901.07714/) (2019)
- [Optimization over Nonnegative and Convex Polynomials With and Without Semidefinite Programming](https://mlsanity.com/paper/1806.06996/) (2018)
- [A multilevel block building algorithm for fast modeling generalized separable systems](https://mlsanity.com/paper/1706.02281/) (2017)
- [SGP-DT: Semantic Genetic Programming Based on Dynamic Targets](https://mlsanity.com/paper/2001.11535/) (2020)

## Built on by (most cited first, 27 of 27)

- [Discovering Causal Relations and Equations from Data](https://mlsanity.com/paper/2305.13341/) (2023)
- [A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge](https://mlsanity.com/paper/2209.06257/) (2022)
- [Controllable Neural Symbolic Regression](https://mlsanity.com/paper/2304.10336/) (2023)
- [Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression](https://mlsanity.com/paper/2302.00773/) (2023)
- [SEVerA: Verified Synthesis of Self-Evolving Agents](https://mlsanity.com/paper/2603.25111/) (2026)
- [In Context Learning and Reasoning for Symbolic Regression with Large Language Models](https://mlsanity.com/paper/2410.17448/) (2024)
- [LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression](https://mlsanity.com/paper/2505.18602/) (2025)
- [Symbolic Regression as Feature Engineering Method for Machine and Deep Learning Regression Tasks](https://mlsanity.com/paper/2311.06028/) (2023)
- [A Comparison of Recent Algorithms for Symbolic Regression to Genetic Programming](https://mlsanity.com/paper/2406.03585/) (2024)
- [Improving Genetic Programming for Symbolic Regression with Equality Graphs](https://mlsanity.com/paper/2501.17848/) (2025)
- [Prediction Intervals and Confidence Regions for Symbolic Regression Models based on Likelihood Profiles](https://mlsanity.com/paper/2209.06454/) (2022)
- [Shape-constrained Symbolic Regression with NSGA-III](https://mlsanity.com/paper/2209.13851/) (2022)
- [Interpretability in Symbolic Regression: a benchmark of Explanatory Methods using the Feynman data set](https://mlsanity.com/paper/2404.05908/) (2024)
- [Incorporating Background Knowledge in Symbolic Regression using a Computer Algebra System](https://mlsanity.com/paper/2301.11919/) (2023)
- [Shape Constraints in Symbolic Regression using Penalized Least Squares](https://mlsanity.com/paper/2405.20800/) (2024)
- [Alleviating Overfitting in Transformation-Interaction-Rational Symbolic Regression with Multi-Objective Optimization](https://mlsanity.com/paper/2501.01905/) (2025)
- [rEGGression: an Interactive and Agnostic Tool for the Exploration of Symbolic Regression Models](https://mlsanity.com/paper/2501.17859/) (2025)
- [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025)
- [Pulling the Carpet Below the Learner's Feet: Genetic Algorithm To Learn Ensemble Machine Learning Model During Concept Drift](https://mlsanity.com/paper/2412.09035/) (2024)
- [Parsing the Language of Expression: Enhancing Symbolic Regression with Domain-Aware Symbolic Priors](https://mlsanity.com/paper/2503.09592/) (2025)
- [Neuro-Evolutionary Approach to Physics-Aware Symbolic Regression](https://mlsanity.com/paper/2504.16503/) (2025)
- [Equality Graph Assisted Symbolic Regression](https://mlsanity.com/paper/2511.01009/) (2025)
- [Identification of Empirical Constitutive Models for Age-Hardenable Aluminium Alloy and High-Chromium Martensitic Steel Using Symbolic Regression](https://mlsanity.com/paper/2511.08424/) (2025)
- [Comparing Shape-Constrained Regression Algorithms for Data Validation](https://mlsanity.com/paper/2209.09602/) (2022)
- [Can Synthetic Data Improve Symbolic Regression Extrapolation Performance?](https://mlsanity.com/paper/2511.22794/) (2025)
- [Automatic Ordinary Differential Equations Discovery For Biological Systems Using Large Language Model Powered Agentic System](https://mlsanity.com/paper/2607.13608/) (2026)
- [AutoSR: Automatic Symbolic Regression by Searching Research States](https://mlsanity.com/paper/2608.16876/) (2026)

## Abstract (arXiv)

We investigate the addition of constraints on the function image and its derivatives for the incorporation of prior knowledge in symbolic regression. The approach is called shape-constrained symbolic regression and allows us to enforce e.g. monotonicity of the function over selected inputs. The aim is to find models which conform to expected behaviour and which have improved extrapolation capabilities. We demonstrate the feasibility of the idea and propose and compare two evolutionary algorithms for shape-constrained symbolic regression: i) an extension of tree-based genetic programming which discards infeasible solutions in the selection step, and ii) a two population evolutionary algorithm that separates the feasible from the infeasible solutions. In both algorithms we use interval arithmetic to approximate bounds for models and their partial derivatives. The algorithms are tested on a set of 19 synthetic and four real-world regression problems. Both algorithms are able to identify models which conform to shape constraints which is not the case for the unmodified symbolic regression algorithms. However, the predictive accuracy of models with constraints is worse on the training set and the test set. Shape-constrained polynomial regression produces the best results for the test set but also significantly larger models.