# Reliable Reasoning Beyond Natural Language

arXiv: https://arxiv.org/abs/2407.11373 · HTML: https://mlsanity.com/paper/2407.11373/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 28 of its 41 references that are in the map.
- First posted on arXiv 16 Jul 2024.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 28 of 28)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [LaMDA: Language Models for Dialog Applications](https://mlsanity.com/paper/2201.08239/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks](https://mlsanity.com/paper/2211.12588/) (2022)
- [Are NLP Models really able to Solve Simple Math Word Problems?](https://mlsanity.com/paper/2103.07191/) (2021)
- [Galactica: A Large Language Model for Science](https://mlsanity.com/paper/2211.09085/) (2022)
- [PAL: Program-aided Language Models](https://mlsanity.com/paper/2211.10435/) (2022)
- [Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems](https://mlsanity.com/paper/1705.04146/) (2017)
- [Faith and Fate: Limits of Transformers on Compositionality](https://mlsanity.com/paper/2305.18654/) (2023)
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](https://mlsanity.com/paper/2302.12813/) (2023)
- [ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Language](https://mlsanity.com/paper/2012.13048/) (2020)
- [Transformers as Soft Reasoners over Language](https://mlsanity.com/paper/2002.05867/) (2020)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [FOLIO: Natural Language Reasoning with First-Order Logic](https://mlsanity.com/paper/2209.00840/) (2022)
- [LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers](https://mlsanity.com/paper/2310.15164/) (2023)
- [The Lean mathematical library](https://mlsanity.com/paper/1910.09336/) (2019)
- [On the Paradox of Learning to Reason from Data](https://mlsanity.com/paper/2205.11502/) (2022)
- [Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2107.02794/) (2021)

## Built on by (most cited first, 14 of 14)

- [Explainable Rule Application via Structured Prompting: A Neural-Symbolic Approach](https://mlsanity.com/paper/2506.16335/) (2025)
- [MathGAP: Out-of-Distribution Evaluation on Problems with Arbitrarily Complex Proofs](https://mlsanity.com/paper/2410.13502/) (2024)
- [ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMs](https://mlsanity.com/paper/2506.15211/) (2025)
- [Equitable Access to Justice: Logical LLMs Show Promise](https://mlsanity.com/paper/2410.09904/) (2024)
- [Creation of AI-driven Smart Spaces for Enhanced Indoor Environments -- A Survey](https://mlsanity.com/paper/2412.14708/) (2024)
- [Models Can and Should Embrace the Communicative Nature of Human-Generated Math](https://mlsanity.com/paper/2409.17005/) (2024)
- [Visualizing Thought: Conceptual Diagrams Enable Robust Planning in LMMs](https://mlsanity.com/paper/2503.11790/) (2025)
- [π-CoT: Prolog-Initialized Chain-of-Thought Prompting for Multi-Hop Question-Answering](https://mlsanity.com/paper/2506.20642/) (2025)
- [Structured Decomposition for LLM Reasoning: Cross-Domain Validation and Semantic Web Integration](https://mlsanity.com/paper/2601.01609/) (2026)
- [Autonomous Business System via Neuro-symbolic AI](https://mlsanity.com/paper/2601.15599/) (2026)
- [PrologMCP: A Standardized Prolog Tool Interface for LLM Agents](https://mlsanity.com/paper/2606.14935/) (2026)
- [NeSy-RAG: Neuro-Symbolic RAG for Explainable Question Answering](https://mlsanity.com/paper/2608.06292/) (2026)
- [Enhancing Mathematical Reasoning in LLMs with Background Operators](https://mlsanity.com/paper/2412.04110/) (2024)
- [CBCL: Safe Self-Extending Agent Communication](https://mlsanity.com/paper/2604.14512/) (2026)

## Abstract (arXiv)

Despite their linguistic competence, Large Language Models (LLMs) often struggle to reason reliably and flexibly. To identify these shortcomings, we introduce the Non-Linear Reasoning (NLR) dataset, a collection of 55 unique, hand-designed problems that target reasoning bottlenecks arising from the sequential prediction paradigm of LLMs and the inherently linear nature of natural language. NLR tasks require iterative updates, backtracking, and reasoning across multiple parallel chains of thought but only basic arithmetic to solve. To address these limitations, we propose a neurosymbolic reasoning approach that integrates Prolog, a symbolic reasoning engine, into the inference pipeline of LLMs. This division of labor shifts the LLM's task from iterative computations to inferring all information, explicit or implied through common sense, and encoding it as logical code. Our method yields large and robust performance gains across the GSM8k and BIG-bench Navigate benchmarks and achieves near-perfect accuracy on NLR problems, maintaining robustness even as variable interdependence - the number of other variables on which the value of a single variable depends - increases.