# Learning Symbolic Expressions via Gumbel-Max Equation Learner Networks

arXiv: https://arxiv.org/abs/2012.06921 · HTML: https://mlsanity.com/paper/2012.06921/

## Key facts

- Cited by 8 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 12 Dec 2020.

## Built on by (most cited first, 8 of 8)

- [SymbolicGPT: A Generative Transformer Model for Symbolic Regression](https://mlsanity.com/paper/2106.14131/) (2021)
- [Deep Differentiable Logic Gate Networks](https://mlsanity.com/paper/2210.08277/) (2022)
- [Learning Interpretable Differentiable Logic Networks](https://mlsanity.com/paper/2407.04168/) (2024)
- [MetaPhysiCa: OOD Robustness in Physics-informed Machine Learning](https://mlsanity.com/paper/2303.03181/) (2023)
- [Scalable Interconnect Learning in Boolean Networks](https://mlsanity.com/paper/2507.02585/) (2025)
- [A Comparative Study of Neurosymbolic AI Approaches to Interpretable Logical Reasoning](https://mlsanity.com/paper/2508.03366/) (2025)
- [A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression](https://mlsanity.com/paper/2407.00065/) (2024)
- [Deep Metric Tensor Regularized Policy Gradient](https://mlsanity.com/paper/2305.11017/) (2023)

## Abstract (arXiv)

Most of the neural networks (NNs) learned via state-of-the-art machine learning techniques are black-box models. For a widespread success of machine learning in science and engineering, it is important to develop new NN architectures to effectively extract high-level mathematical knowledge from complex datasets. Motivated by this understanding, this paper develops a new NN architecture called the Gumbel-Max Equation Learner (GMEQL) network. Different from previously proposed Equation Learner (EQL) networks, GMEQL applies continuous relaxation to the network structure via the Gumbel-Max trick and introduces two types of trainable parameters: structure parameters and regression parameters. This paper also proposes a two-stage training process with new techniques to train structure parameters in both online and offline settings based on an elite repository. On 8 benchmark symbolic regression problems, GMEQL is experimentally shown to outperform several cutting-edge machine learning approaches.