# Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large p

arXiv: https://arxiv.org/abs/2410.13681 · HTML: https://mlsanity.com/paper/2410.13681/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 54 references that are in the map.
- First posted on arXiv 17 Oct 2024.

## Builds on (most cited first, 27 of 27)

- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [OpenML: networked science in machine learning](https://mlsanity.com/paper/1407.7722/) (2014)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Contemporary Symbolic Regression Methods and their Relative Performance](https://mlsanity.com/paper/2107.14351/) (2021)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [AI Feynman 2.0: Pareto-optimal symbolic regression exploiting graph modularity](https://mlsanity.com/paper/2006.10782/) (2020)
- [Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection](https://mlsanity.com/paper/1610.02351/) (2016)
- [Interpretable Scientific Discovery with Symbolic Regression: A Review](https://mlsanity.com/paper/2211.10873/) (2022)
- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [Rediscovering orbital mechanics with machine learning](https://mlsanity.com/paper/2202.02306/) (2022)
- [Symbolic Regression is NP-hard](https://mlsanity.com/paper/2207.01018/) (2022)
- [PMLB v1.0: An open source dataset collection for benchmarking machine learning methods](https://mlsanity.com/paper/2012.00058/) (2020)
- [Deep symbolic regression for physics guided by units constraints: toward the automated discovery of physical laws](https://mlsanity.com/paper/2303.03192/) (2023)
- [Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery](https://mlsanity.com/paper/2206.10540/) (2022)
- [Deep Generative Symbolic Regression with Monte-Carlo-Tree-Search](https://mlsanity.com/paper/2302.11223/) (2023)
- [Learning concise representations for regression by evolving networks of trees](https://mlsanity.com/paper/1807.00981/) (2018)
- [A computational framework for physics-informed symbolic regression with straightforward integration of domain knowledge](https://mlsanity.com/paper/2209.06257/) (2022)
- [Fast, accurate, and transferable many-body interatomic potentials by symbolic regression](https://mlsanity.com/paper/1904.01095/) (2019)
- [A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data](https://mlsanity.com/paper/2309.13705/) (2023)
- [Variable selection for BART: An application to gene regulation](https://mlsanity.com/paper/1310.4887/) (2013)
- [Variable Selection with ABC Bayesian Forests](https://mlsanity.com/paper/1806.02304/) (2018)
- [Constructing Parsimonious Analytic Models for Dynamic Systems via Symbolic Regression](https://mlsanity.com/paper/1903.11483/) (2019)
- [Predicting Friction System Performance with Symbolic Regression and Genetic Programming with Factor Variables](https://mlsanity.com/paper/2107.09484/) (2021)
- [Symbolic Regression for Space Applications: Differentiable Cartesian Genetic Programming Powered by Multi-objective Memetic Algorithms](https://mlsanity.com/paper/2206.06213/) (2022)
- [Operator-induced structural variable selection for identifying materials genes](https://mlsanity.com/paper/2110.10195/) (2021)
- [Ranking Perspective for Tree-based Methods with Applications to Symbolic Feature Selection](https://mlsanity.com/paper/2410.02623/) (2024)

## Built on by (most cited first, 3 of 3)

- [Beyond Accuracy and Complexity: The Effective Information Criterion for Structurally Stable Symbolic Regression](https://mlsanity.com/paper/2509.21780/) (2025)
- [Symbolic Density Estimation for Discrete Distributions](https://mlsanity.com/paper/2605.21813/) (2026)
- [LLM Sparsity Prior for Robust Feature Selection](https://mlsanity.com/paper/2605.23102/) (2026)

## Abstract (arXiv)

Symbolic regression (SR) is a powerful technique for discovering symbolic expressions that characterize nonlinear relationships in data, gaining increasing attention for its interpretability, compactness, and robustness. However, existing SR methods do not scale to datasets with a large number of input variables (referred to as extreme-scale SR), which is common in modern scientific applications. This ``large p'' setting, often accompanied by measurement error, leads to slow performance of SR methods and overly complex expressions that are difficult to interpret. To address this scalability challenge, we propose a method called PAN+SR, which combines a key idea of ab initio nonparametric variable selection with SR to efficiently pre-screen large input spaces and reduce search complexity while maintaining accuracy. The use of nonparametric methods eliminates model misspecification, supporting a strategy called parametric-assisted nonparametric (PAN). We also extend SRBench, an open-source benchmarking platform, by incorporating high-dimensional regression problems with various signal-to-noise ratios. Our results demonstrate that PAN+SR consistently enhances the performance of 19 contemporary SR methods, enabling several to achieve state-of-the-art performance on these challenging datasets.