# Fine-Tuning BERTs for Definition Extraction from Mathematical Text

arXiv: https://arxiv.org/abs/2406.13827 · HTML: https://mlsanity.com/paper/2406.13827/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 5 of its 10 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 5 of 5)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education](https://mlsanity.com/paper/2106.07340/) (2021)
- [DSC IIT-ISM at SemEval-2020 Task 6: Boosting BERT with Dependencies for Definition Extraction](https://mlsanity.com/paper/2009.08180/) (2020)
- [Parmesan: mathematical concept extraction for education](https://mlsanity.com/paper/2307.06699/) (2023)
- [MathGloss: Building mathematical glossaries from text](https://mlsanity.com/paper/2311.12649/) (2023)

## Built on by (most cited first, 1 of 1)

- [MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training](https://mlsanity.com/paper/2502.20855/) (2025)

## Abstract (arXiv)

In this paper, we fine-tuned three pre-trained BERT models on the task of "definition extraction" from mathematical English written in LaTeX. This is presented as a binary classification problem, where either a sentence contains a definition of a mathematical term or it does not. We used two original data sets, "Chicago" and "TAC," to fine-tune and test these models. We also tested on WFMALL, a dataset presented by Vanetik and Litvak in 2021 and compared the performance of our models to theirs. We found that a high-performance Sentence-BERT transformer model performed best based on overall accuracy, recall, and precision metrics, achieving comparable results to the earlier models with less computational effort.