Models Can and Should Embrace the Communicative Nature of Human-Generated Math
Not on the map. Why?
Abstract (arXiv)
Math is constructed by people for people: just as natural language corpora reflect not just propositions but the communicative goals of language users, the math data that models are trained on reflects not just idealized mathematical entities but rich communicative intentions. While there are important advantages to treating math in a purely symbolic manner, we here hypothesize that there are benefits to treating math as situated linguistic communication and that language models are well suited for this goal, in ways that are not fully appreciated. We illustrate these points with two case studies. First, we ran an experiment in which we found that language models interpret the equals sign in a humanlike way -- generating systematically different word problems for the same underlying equation arranged in different ways. Second, we found that language models prefer proofs to be ordered in naturalistic ways, even though other orders would be logically equivalent. We advocate for AI systems that learn from and represent the communicative intentions latent in human-generated math.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 51 references that are in the map. First posted on arXiv 25 Sep 2024.
Builds on
20 of 51 references.
References · 20
- 2024Qwen2 Technical Reportcited by 2,076
- 2021Are NLP Models really able to Solve Simple Math Word Problems?cited by 861
- 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 820
- 2022PAL: Program-aided Language Modelscited by 635
- 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 399
- 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 257
- 2019Assessing BERT's Syntactic Abilitiescited by 191
- 2023Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solvecited by 175
- 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Proverscited by 161
- 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
- 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 100
- 2024Building Machines that Learn and Think with Peoplecited by 73
8 more
- 2024Premise Order Matters in Reasoning with Large Language Modelscited by 71
- 2022A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Modelscited by 68
- 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
- 2023The Larger They Are, the Harder They Fail: Language Models do not Recognize Identifier Swaps in Pythoncited by 29
- 2024Do Language Models Exhibit the Same Cognitive Biases in Problem Solving as Human Learners?cited by 19
- 2024Reliable Reasoning Beyond Natural Languagecited by 14
- 2023AI for Mathematics: A Cognitive Science Perspectivecited by 10
- 2023Exploring the Reversal Curse and Other Deductive Logical Reasoning in BERT and GPT-Based Large Language Modelscited by 5
Built on by
Citing papers per year: 2024: 1.
1 paper.