# MAMUT: A Novel Framework for Modifying Mathematical Formulas for the Generation of Specialized Datasets for Language Model Training

arXiv: https://arxiv.org/abs/2502.20855 · HTML: https://mlsanity.com/paper/2502.20855/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 55 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://mlsanity.com/paper/2002.08155/) (2020)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Analysing Mathematical Reasoning Abilities of Neural Models](https://mlsanity.com/paper/1904.01557/) (2019)
- [We-Math: Does Your Large Multimodal Model Achieve Human-like Mathematical Reasoning?](https://mlsanity.com/paper/2407.01284/) (2024)
- [MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs](https://mlsanity.com/paper/2402.16352/) (2024)
- [MathBERT: A Pre-Trained Model for Mathematical Formula Understanding](https://mlsanity.com/paper/2105.00377/) (2021)
- [MathOdyssey: Benchmarking Mathematical Problem-Solving Skills in Large Language Models Using Odyssey Math Data](https://mlsanity.com/paper/2406.18321/) (2024)
- [MathBERT: A Pre-trained Language Model for General NLP Tasks in Mathematics Education](https://mlsanity.com/paper/2106.07340/) (2021)
- [HARDMath: A Benchmark Dataset for Challenging Problems in Applied Mathematics](https://mlsanity.com/paper/2410.09988/) (2024)
- [Mathematical Language Models: A Survey](https://mlsanity.com/paper/2312.07622/) (2023)
- [Utilizing BERT for Information Retrieval: Survey, Applications, Resources, and Challenges](https://mlsanity.com/paper/2403.00784/) (2024)
- [A Symbolic Framework for Evaluating Mathematical Reasoning and Generalisation with Transformers](https://mlsanity.com/paper/2305.12563/) (2023)
- [InfinityMATH: A Scalable Instruction Tuning Dataset in Programmatic Mathematical Reasoning](https://mlsanity.com/paper/2408.07089/) (2024)
- [Investigating the Effectiveness of ChatGPT in Mathematical Reasoning and Problem Solving: Evidence from the Vietnamese National High School Graduation Examination](https://mlsanity.com/paper/2306.06331/) (2023)
- [Tree-Based Representation and Generation of Natural and Mathematical Language](https://mlsanity.com/paper/2302.07974/) (2023)
- [Neural Machine Translation for Mathematical Formulae](https://mlsanity.com/paper/2305.16433/) (2023)
- [Challenging Instances are Worth Learning: Generating Valuable Negative Samples for Response Selection Training](https://mlsanity.com/paper/2109.06538/) (2021)
- [Mathematical Information Retrieval: Search and Question Answering](https://mlsanity.com/paper/2408.11646/) (2024)
- [Fine-Tuning BERTs for Definition Extraction from Mathematical Text](https://mlsanity.com/paper/2406.13827/) (2024)

## Built on by (most cited first, 1 of 1)

- [EuroBERT: Scaling Multilingual Encoders for European Languages](https://mlsanity.com/paper/2503.05500/) (2025)

## Abstract (arXiv)

Mathematical formulas are a fundamental and widely used component in various scientific fields, serving as a universal language for expressing complex concepts and relationships. While state-of-the-art transformer models excel in processing and understanding natural language, they encounter challenges with mathematical notation, which involves a complex structure and diverse representations. This study focuses on the development of specialized training datasets to enhance the encoding of mathematical content. We introduce Math Mutator (MAMUT), a framework capable of generating equivalent and falsified versions of a given mathematical formula in LaTeX notation, effectively capturing the mathematical variety in notation of the same concept. Based on MAMUT, we have generated four large mathematical datasets containing diverse notation. Experiments show that models trained on these datasets exhibit new SoTA performance on mathematical retrieval tasks. We publish our code, generated datasets, and pretrained mathematical models: https://github.com/aieng-lab/math-mutator.