# Leveraging Large Language Models to Generate Answer Set Programs

arXiv: https://arxiv.org/abs/2307.07699 · HTML: https://mlsanity.com/paper/2307.07699/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 45 references that are in the map.
- First posted on arXiv 15 Jul 2023.

## Builds on (most cited first, 7 of 7)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Scaling Language Models: Methods, Analysis & Insights from Training Gopher](https://mlsanity.com/paper/2112.11446/) (2021)
- [PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Change](https://mlsanity.com/paper/2206.10498/) (2022)
- [Selection-Inference: Exploiting Large Language Models for Interpretable Logical Reasoning](https://mlsanity.com/paper/2205.09712/) (2022)
- [Improving Coherence and Consistency in Neural Sequence Models with Dual-System, Neuro-Symbolic Reasoning](https://mlsanity.com/paper/2107.02794/) (2021)

## Built on by (most cited first, 30 of 33)

- [Argumentative Large Language Models for Explainable and Contestable Claim Verification](https://mlsanity.com/paper/2405.02079/) (2024)
- [NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domains](https://mlsanity.com/paper/2503.00870/) (2025)
- [BlendRL: A Framework for Merging Symbolic and Neural Policy Learning](https://mlsanity.com/paper/2410.11689/) (2024)
- [The Role of Foundation Models in Neuro-Symbolic Learning and Reasoning](https://mlsanity.com/paper/2402.01889/) (2024)
- [VERUS-LM: a Versatile Framework for Combining LLMs with Symbolic Reasoning](https://mlsanity.com/paper/2501.14540/) (2025)
- [FINEREASON: Evaluating and Improving LLMs' Deliberate Reasoning through Reflective Puzzle Solving](https://mlsanity.com/paper/2502.20238/) (2025)
- [DeiSAM: Segment Anything with Deictic Prompting](https://mlsanity.com/paper/2402.14123/) (2024)
- [A Neurosymbolic Approach to Natural Language Formalization and Verification](https://mlsanity.com/paper/2511.09008/) (2025)
- [Towards Automatic Composition of ASP Programs from Natural Language Specifications](https://mlsanity.com/paper/2403.04541/) (2024)
- [Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMs](https://mlsanity.com/paper/2411.18564/) (2024)
- [NeSyPr: Neurosymbolic Proceduralization For Efficient Embodied Reasoning](https://mlsanity.com/paper/2510.19429/) (2025)
- [DCP-Bench-Open: Evaluating LLMs for Constraint Modelling of Discrete Combinatorial Problems](https://mlsanity.com/paper/2506.06052/) (2025)
- [Question Answering with LLMs and Learning from Answer Sets](https://mlsanity.com/paper/2509.16590/) (2025)
- [Organizing a Society of Language Models: Structures and Mechanisms for Enhanced Collective Intelligence](https://mlsanity.com/paper/2405.03825/) (2024)
- [TIC: Translate-Infer-Compile for accurate "text to plan" using LLMs and Logical Representations](https://mlsanity.com/paper/2402.06608/) (2024)
- [Visual Graph Question Answering with ASP and LLMs for Language Parsing](https://mlsanity.com/paper/2502.09211/) (2025)
- [Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysis](https://mlsanity.com/paper/2509.07122/) (2025)
- [Predicate Renaming via Large Language Models](https://mlsanity.com/paper/2510.25517/) (2025)
- [Extending Answer Set Programming with Rational Numbers](https://mlsanity.com/paper/2312.04249/) (2023)
- [textttSEM-CTRL: Semantically Controlled Decoding](https://mlsanity.com/paper/2503.01804/) (2025)
- [Towards a Neurosymbolic Reasoning System Grounded in Schematic Representations](https://mlsanity.com/paper/2509.03644/) (2025)
- [Interpretability Framework for LLMs in Undergraduate Calculus](https://mlsanity.com/paper/2510.17910/) (2025)
- [Bridging Natural Language and ASP: A Hybrid Approach Using LLMs and AMR Parsing](https://mlsanity.com/paper/2511.08715/) (2025)
- [Modeling Next-Token Prediction as Left-Nested Intuitionistic Implication](https://mlsanity.com/paper/2601.19915/) (2026)
- [ASP-Bench: From Natural Language to Logic Programs](https://mlsanity.com/paper/2602.01171/) (2026)
- [Cross-Domain Demo-to-Code via Neurosymbolic Counterfactual Reasoning](https://mlsanity.com/paper/2603.18495/) (2026)
- [Streamliners for Answer Set Programming](https://mlsanity.com/paper/2604.19251/) (2026)
- [Early-Stage Product Line Validation Using LLMs: A Study on Semi-Formal Blueprint Analysis](https://mlsanity.com/paper/2604.20523/) (2026)
- [Distilling Answer-Set Programming Rules from LLMs for Neurosymbolic Visual Question Answering](https://mlsanity.com/paper/2606.03269/) (2026)
- [Driving, Fast or Slow? Neuro-Symbolic Guidance for Motion Prediction in Multi-Modal Ground Mobility](https://mlsanity.com/paper/2606.15251/) (2026)

## Abstract (arXiv)

Large language models (LLMs), such as GPT-3 and GPT-4, have demonstrated exceptional performance in various natural language processing tasks and have shown the ability to solve certain reasoning problems. However, their reasoning capabilities are limited and relatively shallow, despite the application of various prompting techniques. In contrast, formal logic is adept at handling complex reasoning, but translating natural language descriptions into formal logic is a challenging task that non-experts struggle with. This paper proposes a neuro-symbolic method that combines the strengths of large language models and answer set programming. Specifically, we employ an LLM to transform natural language descriptions of logic puzzles into answer set programs. We carefully design prompts for an LLM to convert natural language descriptions into answer set programs in a step by step manner. Surprisingly, with just a few in-context learning examples, LLMs can generate reasonably complex answer set programs. The majority of errors made are relatively simple and can be easily corrected by humans, thus enabling LLMs to effectively assist in the creation of answer set programs.