# Learning concise representations for regression by evolving networks of trees

arXiv: https://arxiv.org/abs/1807.00981 · HTML: https://mlsanity.com/paper/1807.00981/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jul 2018.

## Built on by (most cited first, 30 of 33)

- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [LLM and Simulation as Bilevel Optimizers: A New Paradigm to Advance Physical Scientific Discovery](https://mlsanity.com/paper/2405.09783/) (2024)
- [Symbolic Regression is NP-hard](https://mlsanity.com/paper/2207.01018/) (2022)
- [ODEFormer: Symbolic Regression of Dynamical Systems with Transformers](https://mlsanity.com/paper/2310.05573/) (2023)
- [Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery](https://mlsanity.com/paper/2206.10540/) (2022)
- [A Survey of Deep Learning and Foundation Models for Time Series Forecasting](https://mlsanity.com/paper/2401.13912/) (2024)
- [Predicting Ordinary Differential Equations with Transformers](https://mlsanity.com/paper/2307.12617/) (2023)
- [Taylor Genetic Programming for Symbolic Regression](https://mlsanity.com/paper/2205.09751/) (2022)
- [Symbolic regression via MDLformer-guided search: from minimizing prediction error to minimizing description length](https://mlsanity.com/paper/2411.03753/) (2024)
- [Call for Action: towards the next generation of symbolic regression benchmark](https://mlsanity.com/paper/2505.03977/) (2025)
- [A Neural-Guided Dynamic Symbolic Network for Exploring Mathematical Expressions from Data](https://mlsanity.com/paper/2309.13705/) (2023)
- [A Neural Symbolic Model for Space Physics](https://mlsanity.com/paper/2503.07994/) (2025)
- [Efficient Symbolic Policy Learning with Differentiable Symbolic Expression](https://mlsanity.com/paper/2311.02104/) (2023)
- [GENSR: Symbolic Regression Based in Equation Generative Space](https://mlsanity.com/paper/2602.20557/) (2026)
- [GINN-LP: A Growing Interpretable Neural Network for Discovering Multivariate Laurent Polynomial Equations](https://mlsanity.com/paper/2312.10913/) (2023)
- [LLM-Meta-SR: In-Context Learning for Evolving Selection Operators in Symbolic Regression](https://mlsanity.com/paper/2505.18602/) (2025)
- [Transformation-Interaction-Rational Representation for Symbolic Regression](https://mlsanity.com/paper/2205.06807/) (2022)
- [Modeling Latent Non-Linear Dynamical System over Time Series](https://mlsanity.com/paper/2412.08114/) (2024)
- [Bridging the Domain Gap in Equation Distillation with Reinforcement Feedback](https://mlsanity.com/paper/2505.15572/) (2025)
- [Probabilistic Lexicase Selection](https://mlsanity.com/paper/2305.11681/) (2023)
- [Ab Initio Nonparametric Variable Selection for Scalable Symbolic Regression with Large p](https://mlsanity.com/paper/2410.13681/) (2024)
- [Online Symbolic Regression with Informative Query](https://mlsanity.com/paper/2302.10539/) (2023)
- [Inexact Simplification of Symbolic Regression Expressions with Locality-sensitive Hashing](https://mlsanity.com/paper/2404.05898/) (2024)
- [Information Fusion via Symbolic Regression: A Tutorial in the Context of Human Health](https://mlsanity.com/paper/2306.00153/) (2023)
- [Sharpness-Aware Minimization for Evolutionary Feature Construction in Regression](https://mlsanity.com/paper/2405.06869/) (2024)
- [Discovering Car-following Dynamics from Trajectory Data through Deep Learning](https://mlsanity.com/paper/2408.00251/) (2024)
- [Towards symbolic regression for interpretable clinical decision scores](https://mlsanity.com/paper/2512.07961/) (2025)
- [Zoetrope Genetic Programming for Regression](https://mlsanity.com/paper/2102.13388/) (2021)
- [UniSymNet: A Unified Symbolic Network Guided by Transformer](https://mlsanity.com/paper/2505.06091/) (2025)
- [Iterative Learning of Computable Phenotypes for Treatment Resistant Hypertension using Large Language Models](https://mlsanity.com/paper/2508.05581/) (2025)

## Abstract (arXiv)

We propose and study a method for learning interpretable representations for the task of regression. Features are represented as networks of multi-type expression trees comprised of activation functions common in neural networks in addition to other elementary functions. Differentiable features are trained via gradient descent, and the performance of features in a linear model is used to weight the rate of change among subcomponents of each representation. The search process maintains an archive of representations with accuracy-complexity trade-offs to assist in generalization and interpretation. We compare several stochastic optimization approaches within this framework. We benchmark these variants on 100 open-source regression problems in comparison to state-of-the-art machine learning approaches. Our main finding is that this approach produces the highest average test scores across problems while producing representations that are orders of magnitude smaller than the next best performing method (gradient boosting). We also report a negative result in which attempts to directly optimize the disentanglement of the representation result in more highly correlated features.