# CoNSoLe: Convex Neural Symbolic Learning

arXiv: https://arxiv.org/abs/2206.00257 · HTML: https://mlsanity.com/paper/2206.00257/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 1 Jun 2022.

## Builds on (most cited first, 26 of 26)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Regularized Evolution for Image Classifier Architecture Search](https://mlsanity.com/paper/1802.01548/) (2018)
- [AI Feynman: a Physics-Inspired Method for Symbolic Regression](https://mlsanity.com/paper/1905.11481/) (2019)
- [Progressive Neural Architecture Search](https://mlsanity.com/paper/1712.00559/) (2017)
- [Input Convex Neural Networks](https://mlsanity.com/paper/1609.07152/) (2016)
- [Breaking the Curse of Dimensionality with Convex Neural Networks](https://mlsanity.com/paper/1412.8690/) (2014)
- [Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients](https://mlsanity.com/paper/1912.04871/) (2019)
- [Designing Neural Network Architectures using Reinforcement Learning](https://mlsanity.com/paper/1611.02167/) (2016)
- [Deep Learning without Poor Local Minima](https://mlsanity.com/paper/1605.07110/) (2016)
- [A Theoretical Analysis of Deep Q-Learning](https://mlsanity.com/paper/1901.00137/) (2019)
- [Neural Symbolic Regression that Scales](https://mlsanity.com/paper/2106.06427/) (2021)
- [Physics-informed learning of governing equations from scarce data](https://mlsanity.com/paper/2005.03448/) (2020)
- [Learning Equations for Extrapolation and Control](https://mlsanity.com/paper/1806.07259/) (2018)
- [Extrapolation and learning equations](https://mlsanity.com/paper/1610.02995/) (2016)
- [Symbolic Regression via Neural-Guided Genetic Programming Population Seeding](https://mlsanity.com/paper/2111.00053/) (2021)
- [Neural Networks are Convex Regularizers: Exact Polynomial-time Convex Optimization Formulations for Two-layer Networks](https://mlsanity.com/paper/2002.10553/) (2020)
- [On the Expressivity of Markov Reward](https://mlsanity.com/paper/2111.00876/) (2021)
- [Where are we now? A large benchmark study of recent symbolic regression methods](https://mlsanity.com/paper/1804.09331/) (2018)
- [Discovering Nonlinear PDEs from Scarce Data with Physics-encoded Learning](https://mlsanity.com/paper/2201.12354/) (2022)
- [Improving exploration in policy gradient search: Application to symbolic optimization](https://mlsanity.com/paper/2107.09158/) (2021)
- [Piecewise Strong Convexity of Neural Networks](https://mlsanity.com/paper/1810.12805/) (2018)
- [Informed Equation Learning](https://mlsanity.com/paper/2105.06331/) (2021)
- [Distilling Wikipedia mathematical knowledge into neural network models](https://mlsanity.com/paper/2104.05930/) (2021)
- [Incorporating domain knowledge into neural-guided search](https://mlsanity.com/paper/2107.09182/) (2021)
- [Mapping Rule Estimation for Power Flow Analysis in Distribution Grids](https://mlsanity.com/paper/1702.07948/) (2017)

## Built on by (most cited first, 7 of 7)

- [Transformer-based Planning for Symbolic Regression](https://mlsanity.com/paper/2303.06833/) (2023)
- [Symbolic Regression is NP-hard](https://mlsanity.com/paper/2207.01018/) (2022)
- [Latent Mixture of Symmetries for Sample-Efficient Dynamic Learning](https://mlsanity.com/paper/2510.03578/) (2025)
- [Differentiable Genetic Programming for High-dimensional Symbolic Regression](https://mlsanity.com/paper/2304.08915/) (2023)
- [Scalable Neural Symbolic Regression using Control Variables](https://mlsanity.com/paper/2306.04718/) (2023)
- [A Personalised Learning Tool for Physics Undergraduate Students Built On a Large Language Model for Symbolic Regression](https://mlsanity.com/paper/2407.00065/) (2024)
- [LASS-ODE: Scaling ODE Computations to Connect Foundation Models with Dynamical Physical Systems](https://mlsanity.com/paper/2602.01009/) (2026)

## Abstract (arXiv)

Learning the underlying equation from data is a fundamental problem in many disciplines. Recent advances rely on Neural Networks (NNs) but do not provide theoretical guarantees in obtaining the exact equations owing to the non-convexity of NNs. In this paper, we propose Convex Neural Symbolic Learning (CoNSoLe) to seek convexity under mild conditions. The main idea is to decompose the recovering process into two steps and convexify each step. In the first step of searching for right symbols, we convexify the deep Q-learning. The key is to maintain double convexity for both the negative Q-function and the negative reward function in each iteration, leading to provable convexity of the negative optimal Q function to learn the true symbol connections. Conditioned on the exact searching result, we construct a Locally Convex equation Learner (LoCaL) neural network to convexify the estimation of symbol coefficients. With such a design, we quantify a large region with strict convexity in the loss surface of LoCaL for commonly used physical functions. Finally, we demonstrate the superior performance of the CoNSoLe framework over the state-of-the-art on a diverse set of datasets.