mlsanityData to 7 Oct 2026

Large Language Models for Mathematicians

arXiv 2312.04556 ↗First posted 7 Dec 2023cs.CL cs.AI cs.LGcited by 7

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) such as ChatGPT have received immense interest for their general-purpose language understanding and, in particular, their ability to generate high-quality text or computer code. For many professions, LLMs represent an invaluable tool that can speed up and improve the quality of work. In this note, we discuss to what extent they can aid professional mathematicians. We first provide a mathematical description of the transformer model used in all modern language models. Based on recent studies, we then outline best practices and potential issues and report on the mathematical abilities of language models. Finally, we shed light on the potential of LLMs to change how mathematicians work.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 33 of its 44 references that are in the map. First posted on arXiv 7 Dec 2023.

Builds on

33 of 44 references.

References · 33

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2023GPT-4 Technical Reportcited by 18,132
  5. 2022Training language models to follow instructions with human feedbackcited by 15,712
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  9. 2016Layer Normalizationcited by 5,579
  10. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  11. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  12. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
21 more
  1. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  2. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  4. 2023A Survey of Large Language Modelscited by 2,537
  5. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  6. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  7. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  8. 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
  9. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 890
  10. 2021Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Surveycited by 457
  11. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 414
  12. 2023Mathematical Capabilities of ChatGPTcited by 263
  13. 2023GPTs are GPTs: An Early Look at the Labor Market Impact Potential of Large Language Modelscited by 174
  14. 2022Transformer Language Models without Positional Encodings Still Learn Positional Informationcited by 148
  15. 2023Baldur: Whole-Proof Generation and Repair with Large Language Modelscited by 118
  16. 2019Learning to Prove Theorems via Interacting with Proof Assistantscited by 107
  17. 2023Evaluating Language Models for Mathematics through Interactionscited by 69
  18. 2023PanGu-Sigma: Towards Trillion Parameter Language Model with Sparse Heterogeneous Computingcited by 55
  19. 2021Multi-head or Single-head? An Empirical Comparison for Transformer Trainingcited by 15
  20. 2023Is deep learning a useful tool for the pure mathematician?cited by 14
  21. 2023Humans are Still Better than ChatGPT: Case of the IEEEXtreme Competitioncited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 1.

7 papers.

  1. 2024A Survey on Deep Learning for Theorem Provingcited by 55
  2. 2024eCeLLM: Generalizing Large Language Models for E-commerce from Large-scale, High-quality Instruction Datacited by 44
  3. 2025Starjob: Dataset for LLM-Driven Job Shop Schedulingcited by 12
  4. 2025Grammars of Formal Uncertainty: When to Trust LLMs in Automated Reasoning Taskscited by 9
  5. 2025Benford's Curse: Tracing Digit Bias to Numerical Hallucination in LLMscited by 6
  6. 2024A Triumvirate of AI Driven Theoretical Discoverycited by 5
  7. 2026Reliable Use of Lemmas via Eligibility Reasoning and Section-Aware Reinforcement Learningcited by 0