# Efficient Generator of Mathematical Expressions for Symbolic Regression

arXiv: https://arxiv.org/abs/2302.09893 · HTML: https://mlsanity.com/paper/2302.09893/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 43 references that are in the map.
- First posted on arXiv 20 Feb 2023.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Automatic chemical design using a data-driven continuous representation of molecules](https://mlsanity.com/paper/1610.02415/) (2016)
- [Junction Tree Variational Autoencoder for Molecular Graph Generation](https://mlsanity.com/paper/1802.04364/) (2018)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network](https://mlsanity.com/paper/1808.03314/) (2018)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks](https://mlsanity.com/paper/1503.00075/) (2015)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [End-to-end symbolic regression with transformers](https://mlsanity.com/paper/2204.10532/) (2022)
- [pymoo: Multi-objective Optimization in Python](https://mlsanity.com/paper/2002.04504/) (2020)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Syntax-Directed Variational Autoencoder for Structured Data](https://mlsanity.com/paper/1802.08786/) (2018)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [A Bayesian machine scientist to aid in the solution of challenging scientific problems](https://mlsanity.com/paper/2004.12157/) (2020)
- [Deep Symbolic Regression for Recurrent Sequences](https://mlsanity.com/paper/2201.04600/) (2022)
- [GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution](https://mlsanity.com/paper/1611.04051/) (2016)
- [Probabilistic Grammars for Equation Discovery](https://mlsanity.com/paper/2012.00428/) (2020)
- [Learning Continuous Semantic Representations of Symbolic Expressions](https://mlsanity.com/paper/1611.01423/) (2016)
- [Equation Discovery for Nonlinear System Identification](https://mlsanity.com/paper/1907.00821/) (2019)
- [Learning a Generative Model for Validity in Complex Discrete Structures](https://mlsanity.com/paper/1712.01664/) (2017)
- [Natural Language Generation with Neural Variational Models](https://mlsanity.com/paper/1808.09012/) (2018)

## Built on by (most cited first, 13 of 13)

- [SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training](https://mlsanity.com/paper/2310.02227/) (2023)
- [Automated Scientific Discovery: From Equation Discovery to Autonomous Discovery Systems](https://mlsanity.com/paper/2305.02251/) (2023)
- [GENSR: Symbolic Regression Based in Equation Generative Space](https://mlsanity.com/paper/2602.20557/) (2026)
- [AI-Newton: A Concept-Driven Physical Law Discovery System without Prior Physical Knowledge](https://mlsanity.com/paper/2504.01538/) (2025)
- [Approximating the universal thermal climate index using sparse regression with orthogonal polynomials](https://mlsanity.com/paper/2508.11307/) (2025)
- [Probabilistic Regular Tree Priors for Scientific Symbolic Reasoning](https://mlsanity.com/paper/2306.08506/) (2023)
- [Multi-Modal Learning meets Genetic Programming: Analyzing Alignment in Latent Space Optimization](https://mlsanity.com/paper/2604.08324/) (2026)
- [Quantifying Behavioral Dissimilarity Between Mathematical Expressions](https://mlsanity.com/paper/2408.11515/) (2024)
- [SPINEX_ Symbolic Regression: Similarity-based Symbolic Regression with Explainable Neighbors Exploration](https://mlsanity.com/paper/2411.03358/) (2024)
- [Reinforced Symbolic Learning with Logical Constraints for Predicting Turbine Blade Fatigue Life](https://mlsanity.com/paper/2412.03580/) (2024)
- [Neural-Guided Equation Discovery](https://mlsanity.com/paper/2503.16953/) (2025)
- [Bayesian symbolic regression: Automated equation discovery from a physicists' perspective](https://mlsanity.com/paper/2507.19540/) (2025)
- [Synthics: Synthetic Physics-like Datasets for Machine Learning](https://mlsanity.com/paper/2606.06724/) (2026)

## Abstract (arXiv)

We propose an approach to symbolic regression based on a novel variational autoencoder for generating hierarchical structures, HVAE. It combines simple atomic units with shared weights to recursively encode and decode the individual nodes in the hierarchy. Encoding is performed bottom-up and decoding top-down. We empirically show that HVAE can be trained efficiently with small corpora of mathematical expressions and can accurately encode expressions into a smooth low-dimensional latent space. The latter can be efficiently explored with various optimization methods to address the task of symbolic regression. Indeed, random search through the latent space of HVAE performs better than random search through expressions generated by manually crafted probabilistic grammars for mathematical expressions. Finally, EDHiE system for symbolic regression, which applies an evolutionary algorithm to the latent space of HVAE, reconstructs equations from a standard symbolic regression benchmark better than a state-of-the-art system based on a similar combination of deep learning and evolutionary algorithms.vz