# Coupling Large Language Models with Logic Programming for Robust and General Reasoning from Text

arXiv: https://arxiv.org/abs/2307.07696 · HTML: https://mlsanity.com/paper/2307.07696/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 43 of its 120 references that are in the map.
- First posted on arXiv 15 Jul 2023.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 30 of 43)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Inner Monologue: Embodied Reasoning through Planning with Language Models](https://mlsanity.com/paper/2207.05608/) (2022)
- [Universal Transformers](https://mlsanity.com/paper/1807.03819/) (2018)
- [Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language](https://mlsanity.com/paper/2204.00598/) (2022)
- [CLEVRER: CoLlision Events for Video REpresentation and Reasoning](https://mlsanity.com/paper/1910.01442/) (2019)
- [A simple neural network module for relational reasoning](https://mlsanity.com/paper/1706.01427/) (2017)
- [Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks](https://mlsanity.com/paper/1502.05698/) (2015)
- [COMET-ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs](https://mlsanity.com/paper/2010.05953/) (2020)
- [Deep Learning: A Critical Appraisal](https://mlsanity.com/paper/1801.00631/) (2018)
- [Large Language Models are Few-Shot Clinical Information Extractors](https://mlsanity.com/paper/2205.12689/) (2022)
- [Compositional Attention Networks for Machine Reasoning](https://mlsanity.com/paper/1803.03067/) (2018)
- [KnowPrompt: Knowledge-aware Prompt-tuning with Synergistic Optimization for Relation Extraction](https://mlsanity.com/paper/2104.07650/) (2021)
- [Constrained Language Models Yield Few-Shot Semantic Parsers](https://mlsanity.com/paper/2104.08768/) (2021)
- [CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text](https://mlsanity.com/paper/1908.06177/) (2019)
- [Language to Logical Form with Neural Attention](https://mlsanity.com/paper/1601.01280/) (2016)
- [Data Recombination for Neural Semantic Parsing](https://mlsanity.com/paper/1606.03622/) (2016)
- [A Benchmark for Systematic Generalization in Grounded Language Understanding](https://mlsanity.com/paper/2003.05161/) (2020)
- [Graph Neural Networks Meet Neural-Symbolic Computing: A Survey and Perspective](https://mlsanity.com/paper/2003.00330/) (2020)
- [Compositional Semantic Parsing with Large Language Models](https://mlsanity.com/paper/2209.15003/) (2022)
- [Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2107.02794/) (2021)
- [An Improved Baseline for Sentence-level Relation Extraction](https://mlsanity.com/paper/2102.01373/) (2021)
- [Recurrent Relational Networks](https://mlsanity.com/paper/1711.08028/) (2017)
- [Low-Resource Domain Adaptation for Compositional Task-Oriented Semantic Parsing](https://mlsanity.com/paper/2010.03546/) (2020)
- [Grounding Physical Concepts of Objects and Events Through Dynamic Visual Reasoning](https://mlsanity.com/paper/2103.16564/) (2021)
- [StepGame: A New Benchmark for Robust Multi-Hop Spatial Reasoning in Texts](https://mlsanity.com/paper/2204.08292/) (2022)
- [Learning Reasoning Strategies in End-to-End Differentiable Proving](https://mlsanity.com/paper/2007.06477/) (2020)

## Built on by (most cited first, 30 of 59)

- [Understanding the planning of LLM agents: A survey](https://mlsanity.com/paper/2402.02716/) (2024)
- [Mind's Eye of LLMs: Visualization-of-Thought Elicits Spatial Reasoning in Large Language Models](https://mlsanity.com/paper/2404.03622/) (2024)
- [Dynamic Planning with a LLM](https://mlsanity.com/paper/2308.06391/) (2023)
- [TopViewRS: Vision-Language Models as Top-View Spatial Reasoners](https://mlsanity.com/paper/2406.02537/) (2024)
- [On the Prospects of Incorporating Large Language Models (LLMs) in Automated Planning and Scheduling (APS)](https://mlsanity.com/paper/2401.02500/) (2024)
- [Large Language Models for Planning: A Comprehensive and Systematic Survey](https://mlsanity.com/paper/2505.19683/) (2025)
- [Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning](https://mlsanity.com/paper/2410.16162/) (2024)
- [Planetarium: A Rigorous Benchmark for Translating Text to Structured Planning Languages](https://mlsanity.com/paper/2407.03321/) (2024)
- [Robotouille: An Asynchronous Planning Benchmark for LLM Agents](https://mlsanity.com/paper/2502.05227/) (2025)
- [Advancing Spatial Reasoning in Large Language Models: An In-Depth Evaluation and Enhancement Using the StepGame Benchmark](https://mlsanity.com/paper/2401.03991/) (2024)
- [Planning with Multi-Constraints via Collaborative Language Agents](https://mlsanity.com/paper/2405.16510/) (2024)
- [NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains](https://mlsanity.com/paper/2503.00870/) (2025)
- [BlendRL: A Framework for Merging Symbolic and Neural Policy Learning](https://mlsanity.com/paper/2410.11689/) (2024)
- [HyperTree Planning: Enhancing LLM Reasoning via Hierarchical Thinking](https://mlsanity.com/paper/2505.02322/) (2025)
- [The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning](https://mlsanity.com/paper/2402.01889/) (2024)
- [A Human-Like Reasoning Framework for Multi-Phases Planning Task with Large Language Models](https://mlsanity.com/paper/2405.18208/) (2024)
- [VERUS-LM: a Versatile Framework for Combining LLMs with Symbolic Reasoning](https://mlsanity.com/paper/2501.14540/) (2025)
- [SOP-Agent: Empower General Purpose AI Agent with Domain-Specific SOPs](https://mlsanity.com/paper/2501.09316/) (2025)
- [Synergizing Logical Reasoning, Knowledge Management and Collaboration in Multi-Agent LLM System](https://mlsanity.com/paper/2507.02170/) (2025)
- [DeiSAM: Segment Anything with Deictic Prompting](https://mlsanity.com/paper/2402.14123/) (2024)
- [A Neurosymbolic Approach to Natural Language Formalization and Verification](https://mlsanity.com/paper/2511.09008/) (2025)
- [LLASP: Fine-tuning Large Language Models for Answer Set Programming](https://mlsanity.com/paper/2407.18723/) (2024)
- [Enhancing Logical Reasoning in Large Language Models through Graph-based Synthetic Data](https://mlsanity.com/paper/2409.12437/) (2024)
- [Towards Automatic Composition of ASP Programs from Natural Language Specifications](https://mlsanity.com/paper/2403.04541/) (2024)
- [ChatLogic: Integrating Logic Programming with Large Language Models for Multi-Step Reasoning](https://mlsanity.com/paper/2407.10162/) (2024)
- [Towards a Common Framework for Autoformalization](https://mlsanity.com/paper/2509.09810/) (2025)
- [Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs](https://mlsanity.com/paper/2411.18564/) (2024)
- [Adaptive Domain Modeling with Language Models: A Multi-Agent Approach to Task Planning](https://mlsanity.com/paper/2506.19592/) (2025)
- [SpaRC and SpaRP: Spatial Reasoning Characterization and Path Generation for Understanding Spatial Reasoning Capability of Large Language Models](https://mlsanity.com/paper/2406.04566/) (2024)
- [Towards Logically Sound Natural Language Reasoning with Logic-Enhanced Language Model Agents](https://mlsanity.com/paper/2408.16081/) (2024)

## Abstract (arXiv)

While large language models (LLMs), such as GPT-3, appear to be robust and general, their reasoning ability is not at a level to compete with the best models trained for specific natural language reasoning problems. In this study, we observe that a large language model can serve as a highly effective few-shot semantic parser. It can convert natural language sentences into a logical form that serves as input for answer set programs, a logic-based declarative knowledge representation formalism. The combination results in a robust and general system that can handle multiple question-answering tasks without requiring retraining for each new task. It only needs a few examples to guide the LLM's adaptation to a specific task, along with reusable ASP knowledge modules that can be applied to multiple tasks. We demonstrate that this method achieves state-of-the-art performance on several NLP benchmarks, including bAbI, StepGame, CLUTRR, and gSCAN. Additionally, it successfully tackles robot planning tasks that an LLM alone fails to solve.