# Mathematical Reasoning in Latent Space

arXiv: https://arxiv.org/abs/1909.11851 · HTML: https://mlsanity.com/paper/1909.11851/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Sep 2019.

## Built on by (most cited first, 17 of 17)

- [A Survey on Deep Learning for Theorem Proving](https://mlsanity.com/paper/2404.09939/) (2024)
- [MWP-BERT: Numeracy-Augmented Pre-training for Math Word Problem Solving](https://mlsanity.com/paper/2107.13435/) (2021)
- [Can neural networks do arithmetic? A survey on the elementary numerical skills of state-of-the-art deep learning models](https://mlsanity.com/paper/2303.07735/) (2023)
- [Embedding Symbolic Temporal Knowledge into Deep Sequential Models](https://mlsanity.com/paper/2101.11981/) (2021)
- [Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Tasks](https://mlsanity.com/paper/2505.20047/) (2025)
- [AI Descartes: Combining Data and Theory for Derivable Scientific Discovery](https://mlsanity.com/paper/2109.01634/) (2021)
- [Multi-Operational Mathematical Derivations in Latent Space](https://mlsanity.com/paper/2311.01230/) (2023)
- [Generating Symbolic Reasoning Problems with Transformer GANs](https://mlsanity.com/paper/2110.10054/) (2021)
- [Exploring the Common Principal Subspace of Deep Features in Neural Networks](https://mlsanity.com/paper/2110.02863/) (2021)
- [Steering Language Generation: Harnessing Contrastive Expert Guidance and Negative Prompting for Coherent and Diverse Synthetic Data Generation](https://mlsanity.com/paper/2308.07645/) (2023)
- [A Classification of Artificial Intelligence Systems for Mathematics Education](https://mlsanity.com/paper/2107.06015/) (2021)
- [Towards Controllable Natural Language Inference through Lexical Inference Types](https://mlsanity.com/paper/2308.03581/) (2023)
- [Measuring AI Reasoning: A Guide for Researchers](https://mlsanity.com/paper/2605.02442/) (2026)
- [Safety without alignment](https://mlsanity.com/paper/2303.00752/) (2023)
- [Incorporating Graph Attention Mechanism into Geometric Problem Solving Based on Deep Reinforcement Learning](https://mlsanity.com/paper/2403.14690/) (2024)
- [Exploring Internal Numeracy in Language Models: A Case Study on ALBERT](https://mlsanity.com/paper/2404.16574/) (2024)
- [Model-Grounded Symbolic Artificial Intelligence Systems Learning and Reasoning with Model-Grounded Symbolic Artificial Intelligence Systems](https://mlsanity.com/paper/2507.09854/) (2025)

## Abstract (arXiv)

We design and conduct a simple experiment to study whether neural networks can perform several steps of approximate reasoning in a fixed dimensional latent space. The set of rewrites (i.e. transformations) that can be successfully performed on a statement represents essential semantic features of the statement. We can compress this information by embedding the formula in a vector space, such that the vector associated with a statement can be used to predict whether a statement can be rewritten by other theorems. Predicting the embedding of a formula generated by some rewrite rule is naturally viewed as approximate reasoning in the latent space. In order to measure the effectiveness of this reasoning, we perform approximate deduction sequences in the latent space and use the resulting embedding to inform the semantic features of the corresponding formal statement (which is obtained by performing the corresponding rewrite sequence using real formulas). Our experiments show that graph neural networks can make non-trivial predictions about the rewrite-success of statements, even when they propagate predicted latent representations for several steps. Since our corpus of mathematical formulas includes a wide variety of mathematical disciplines, this experiment is a strong indicator for the feasibility of deduction in latent space in general.