# Informed Equation Learning

arXiv: https://arxiv.org/abs/2105.06331 · HTML: https://mlsanity.com/paper/2105.06331/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 13 May 2021.

## Builds on (most cited first, 16 of 16)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Identity Mappings in Deep Residual Networks](https://mlsanity.com/paper/1603.05027/) (2016)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Grammar Variational Autoencoder](https://mlsanity.com/paper/1703.01925/) (2017)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [PDE-Net 2.0: Learning PDEs from Data with A Numeric-Symbolic Hybrid Deep Network](https://mlsanity.com/paper/1812.04426/) (2018)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Integration of Neural Network-Based Symbolic Regression in Deep Learning for Scientific Discovery](https://mlsanity.com/paper/1912.04825/) (2019)
- [Bayesian Symbolic Regression](https://mlsanity.com/paper/1910.08892/) (2019)
- [Learning to Discover Efficient Mathematical Identities](https://mlsanity.com/paper/1406.1584/) (2014)
- [Analytical classical density functionals from an equation learning network](https://mlsanity.com/paper/1910.12752/) (2019)
- [Neural-Guided Symbolic Regression with Asymptotic Constraints](https://mlsanity.com/paper/1901.07714/) (2019)

## Built on by (most cited first, 10 of 10)

- [SymFormer: End-to-end symbolic regression using transformer-based architecture](https://mlsanity.com/paper/2205.15764/) (2022)
- [Toward Physically Plausible Data-Driven Models: A Novel Neural Network Approach to Symbolic Regression](https://mlsanity.com/paper/2302.00773/) (2023)
- [Simultaneous identification of models and parameters of scientific simulators](https://mlsanity.com/paper/2305.15174/) (2023)
- [CoNSoLe: Convex Neural Symbolic Learning](https://mlsanity.com/paper/2206.00257/) (2022)
- [SymbolNet: Neural Symbolic Regression with Adaptive Dynamic Pruning for Compression](https://mlsanity.com/paper/2401.09949/) (2024)
- [Symbolic expression generation via Variational Auto-Encoder](https://mlsanity.com/paper/2301.06064/) (2023)
- [Univariate Skeleton Prediction in Multivariate Systems Using Transformers](https://mlsanity.com/paper/2406.17834/) (2024)
- [Data-driven Discovery of Digital Twins in Biomedical Research](https://mlsanity.com/paper/2508.21484/) (2025)
- [Softly Symbolifying Kolmogorov-Arnold Networks](https://mlsanity.com/paper/2512.07875/) (2025)
- [Scalable Simulation-Based Model Inference with Test-Time Complexity Control](https://mlsanity.com/paper/2603.15292/) (2026)

## Abstract (arXiv)

Distilling data into compact and interpretable analytic equations is one of the goals of science. Instead, contemporary supervised machine learning methods mostly produce unstructured and dense maps from input to output. Particularly in deep learning, this property is owed to the generic nature of simple standard link functions. To learn equations rather than maps, standard non-linearities can be replaced with structured building blocks of atomic functions. However, without strong priors on sparsity and structure, representational complexity and numerical conditioning limit this direct approach. To scale to realistic settings in science and engineering, we propose an informed equation learning system. It provides a way to incorporate expert knowledge about what are permitted or prohibited equation components, as well as a domain-dependent structured sparsity prior. Our system then utilizes a robust method to learn equations with atomic functions exhibiting singularities, as e.g. logarithm and division. We demonstrate several artificial and real-world experiments from the engineering domain, in which our system learns interpretable models of high predictive power.