mlsanityData to 7 Oct 2026

Arithmetic Reasoning with LLM: Prolog Generation & Permutation

arXiv 2405.17893 ↗First posted 28 May 2024cs.CL cs.AIcited by 17

On the map: LLM reasoning → Mathematical reasoning

Abstract (arXiv)

Instructing large language models (LLMs) to solve elementary school math problems has shown great success using Chain of Thought (CoT). However, the CoT approach relies on an LLM to generate a sequence of arithmetic calculations which can be prone to cascaded calculation errors. We hypothesize that an LLM should focus on extracting predicates and generating symbolic formulas from the math problem description so that the underlying calculation can be done via an external code interpreter. We investigate using LLM to generate Prolog programs to solve mathematical questions. Experimental results show that our Prolog-based arithmetic problem-solving outperforms CoT generation in the GSM8K benchmark across three distinct LLMs. In addition, given the insensitive ordering of predicates and symbolic formulas in Prolog, we propose to permute the ground truth predicates for more robust LLM training via data augmentation.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Mathematical reasoning, one of 17 directions in LLM reasoning. Builds on 27 of its 40 references that are in the map. First posted on arXiv 28 May 2024.

Builds on

27 of 40 references.

Closest ancestors · 3

Same direction, newest first

  1. 2023ToRA: A Tool-Integrated Reasoning Agent for Mathematical Problem Solving29 Sep
  2. 2023Scaling Relationship on Learning Mathematical Reasoning with Large Language Models3 Aug
  3. 2022Lila: A Unified Benchmark for Mathematical Reasoning31 Oct

Other references · 24

  1. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  2. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  6. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  7. 2023Mistral 7Bcited by 3,065
  8. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  9. 2022Emergent Abilities of Large Language Modelscited by 2,483
  10. 2023Code Llama: Open Foundation Models for Codecited by 2,105
  11. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  12. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
12 more
  1. 2021Scaling Language Models: Methods, Analysis & Insights from Training Gophercited by 1,199
  2. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  3. 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
  4. 2023Graph of Thoughts: Solving Elaborate Problems with Large Language Modelscited by 873
  5. 2022Large Language Models Can Self-Improvecited by 656
  6. 2022PAL: Program-aided Language Modelscited by 635
  7. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 379
  8. 2023Faithful Chain-of-Thought Reasoningcited by 285
  9. 2023Large Language Model Guided Tree-of-Thoughtcited by 218
  10. 2023Cumulative Reasoning with Large Language Modelscited by 94
  11. 2021Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoningcited by 85
  12. 2023Neuro-Symbolic Integration Brings Causal and Reliable Reasoning Proofscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 9; 2026: 4.

17 papers.

Sort:

By area:

  1. 2025Large Language Models for Computer-Aided Design: A Surveycited by 16
  2. 2024Past, Present, and Future of Sensor-Based Human Activity Recognition Using Wearables: A Surveying Tutorial on a Still Challenging Taskcited by 14
  3. 2024Reversal of Thought: Enhancing Large Language Models with Preference-Guided Reverse Reasoning Warm-upcited by 10
  4. 2025From Reasoning to Code: GRPO Optimization for Underrepresented Languagescited by 10
  5. 2025ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMscited by 9
  6. 2025Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoningcited by 7
  7. 2025Knowledge Augmented Complex Problem Solving with Large Language Models: A Surveycited by 6
  8. 2025A Solver-in-the-Loop Framework for Improving LLMs on Answer Set Programming for Logic Puzzle Solvingcited by 4
  9. 2024Misinforming LLMs: vulnerabilities, challenges and opportunitiescited by 3
  10. 2025Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysiscited by 2
  11. 2026HDSL: A Hierarchical Domain-Specific Language for Structured 3D Indoor Scene Generation and Localized Editing with LLM Agentscited by 2
  12. 2024Khattat: Enhancing Readability and Concept Representation of Semantic Typographycited by 1
5 more
  1. 2025Training Language Models to Use Prolog as a Toolcited by 1
  2. 2026Autonomous Business System via Neuro-symbolic AIcited by 1
  3. 2026AI-rithmeticcited by 1
  4. 2026PrologMCP: A Standardized Prolog Tool Interface for LLM Agentscited by 1
  5. 2025Understanding Benchmark Language Under Weakened Formal Semanticscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploring an LM to generate Prolog Predicates from Mathematics Questions
  2. 2025Evaluating Intermediate Reasoning of Code-Assisted Large Language Models for Mathematics
  3. 2024Enhancing Mathematical Reasoning in LLMs with Background Operators
  4. 2023MathPrompter: Mathematical Reasoning using Large Language Models
  5. 2025Thinking Machines: Mathematical Reasoning in the Age of LLMs
  6. 2024MARIO: MAth Reasoning with code Interpreter Output -- A Reproducible Pipeline
  7. 2023Frugal LMs Trained to Invoke Symbolic Solvers Achieve Parameter-Efficient Arithmetic Reasoning
  8. 2023Learning Multi-Step Reasoning by Solving Arithmetic Tasks