# Syntax Meets Semantics: Understanding Scientific Formulae

arXiv: https://arxiv.org/abs/2608.02457 · HTML: https://mlsanity.com/paper/2608.02457/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 34 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Builds on (most cited first, 12 of 12)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Strategies for Pre-training Graph Neural Networks](https://mlsanity.com/paper/1905.12265/) (2019)
- [MathBERT: A Pre-Trained Model for Mathematical Formula Understanding](https://mlsanity.com/paper/2105.00377/) (2021)
- [MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education](https://mlsanity.com/paper/2106.07340/) (2021)
- [TopicEq: A Joint Topic and Mathematical Equation Model for Scientific Texts](https://mlsanity.com/paper/1902.06034/) (2019)
- [Improving the Representation and Conversion of Mathematical Formulae by Considering their Textual Context](https://mlsanity.com/paper/1804.04956/) (2018)
- [Equation Embeddings](https://mlsanity.com/paper/1803.09123/) (2018)
- [Preliminary Exploration of Formula Embedding for Mathematical Information Retrieval: can mathematical formulae be embedded like a natural language?](https://mlsanity.com/paper/1707.05154/) (2017)
- [SSEmb: A Joint Structural and Semantic Embedding Framework for Mathematical Formula Retrieval](https://mlsanity.com/paper/2508.04162/) (2025)

## Abstract (arXiv)

Scientific formulae are a fundamental component of scholarly communication, yet their dual nature -- as structured syntax and carriers of semantics -- remains underexplored in scholarly information retrieval. Although prior studies show that jointly modeling syntactic and semantic modalities improves retrieval performance, the relationship between their underlying representations has not been systematically investigated. In this work, we empirically study cross-modal correspondence between formula syntax and semantics. We find that their native representation spaces exhibit extremely weak observable correspondence despite strong latent correlation, indicating a substantial representation mismatch between the two modalities. We further evaluate whether this mismatch can be reduced using standard representation learning and alignment techniques. We represent syntactic structure using graph-based encoders and semantic information using text-based encoders, then apply contrastive learning to induce a shared representation space. Results show that the learned alignment substantially improves cross-modal retrieval, suggesting that explicit representation learning can recover correspondence absent from the original representation spaces.