# Semantic Representations of Mathematical Expressions in a Continuous Vector Space

arXiv: https://arxiv.org/abs/2211.08142 · HTML: https://mlsanity.com/paper/2211.08142/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 36 references that are in the map.
- First posted on arXiv 8 Oct 2022.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Skip-Thought Vectors](https://mlsanity.com/paper/1506.06726/) (2015)
- [Deep Learning for Symbolic Mathematics](https://mlsanity.com/paper/1912.01412/) (2019)
- [MathBERT: A Pre-Trained Model for Mathematical Formula Understanding](https://mlsanity.com/paper/2105.00377/) (2021)
- [Enhancing the Transformer with Explicit Relational Encoding for Math Problem Solving](https://mlsanity.com/paper/1910.06611/) (2019)
- [Learning Continuous Semantic Representations of Symbolic Expressions](https://mlsanity.com/paper/1611.01423/) (2016)
- [Compositional Processing Emerges in Neural Networks Solving Math Problems](https://mlsanity.com/paper/2105.08961/) (2021)
- [Equation Embeddings](https://mlsanity.com/paper/1803.09123/) (2018)
- [The Tangent Search Engine: Improved Similarity Metrics and Scalability for Math Formula Search](https://mlsanity.com/paper/1507.06235/) (2015)
- [Preliminary Exploration of Formula Embedding for Mathematical Information Retrieval: can mathematical formulae be embedded like a natural language?](https://mlsanity.com/paper/1707.05154/) (2017)
- [Semantic Sentence Embeddings for Paraphrasing and Text Summarization](https://mlsanity.com/paper/1809.10267/) (2018)

## Built on by (most cited first, 2 of 2)

- [E-Gen: Leveraging E-Graphs to Improve Continuous Representations of Symbolic Expressions](https://mlsanity.com/paper/2501.14951/) (2025)
- [Enhancing Transformer Representations of Symbolic ODE Expressions](https://mlsanity.com/paper/2609.24746/) (2026)

## Abstract (arXiv)

Mathematical notation makes up a large portion of STEM literature, yet finding semantic representations for formulae remains a challenging problem. Because mathematical notation is precise, and its meaning changes significantly with small character shifts, the methods that work for natural text do not necessarily work well for mathematical expressions. This work describes an approach for representing mathematical expressions in a continuous vector space. We use the encoder of a sequence-to-sequence architecture, trained on visually different but mathematically equivalent expressions, to generate vector representations (or embeddings). We compare this approach with a structural approach that considers visual layout to embed an expression and show that our proposed approach is better at capturing mathematical semantics. Finally, to expedite future research, we publish a corpus of equivalent transcendental and algebraic expression pairs.