# Logic-of-Thought: Empowering Large Language Models with Logic Programs for Solving Puzzles in Natural Language

arXiv: https://arxiv.org/abs/2505.16114 · HTML: https://mlsanity.com/paper/2505.16114/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 7 of its 39 references that are in the map.
- First posted on arXiv 22 May 2025.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)

## Builds on (most cited first, 7 of 7)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [DeepSeek-V3 Technical Report](https://mlsanity.com/paper/2412.19437/) (2024)
- [Large Language Models: A Survey](https://mlsanity.com/paper/2402.06196/) (2024)
- [FCoReBench: Can Large Language Models Solve Challenging First-Order Combinatorial Reasoning Problems?](https://mlsanity.com/paper/2402.02611/) (2024)
- [Mathematical Definition and Systematization of Puzzle Rules](https://mlsanity.com/paper/2501.01433/) (2024)

## Built on by (most cited first, 5 of 5)

- [Towards a Common Framework for Autoformalization](https://mlsanity.com/paper/2509.09810/) (2025)
- [PuzzleClone: A DSL-Powered Framework for Synthesizing Verifiable Data](https://mlsanity.com/paper/2508.15180/) (2025)
- [Intermediate Languages Matter: Formal Languages and LLMs affect Neurosymbolic Reasoning](https://mlsanity.com/paper/2509.04083/) (2025)
- [Bridging Natural Language and ASP: A Hybrid Approach Using LLMs and AMR Parsing](https://mlsanity.com/paper/2511.08715/) (2025)
- [SOM: Structured Opponent Modeling for LLM-based Agents via Structural Causal Model](https://mlsanity.com/paper/2605.07301/) (2026)

## Abstract (arXiv)

Solving puzzles in natural language poses a long-standing challenge in AI. While large language models (LLMs) have recently shown impressive capabilities in a variety of tasks, they continue to struggle with complex puzzles that demand precise reasoning and exhaustive search. In this paper, we propose Logic-of-Thought (Logot), a novel framework that bridges LLMs with logic programming to address this problem. Our method leverages LLMs to translate puzzle rules and states into answer set programs (ASPs), the solution of which are then accurately and efficiently inferred by an ASP interpreter. This hybrid approach combines the natural language understanding of LLMs with the precise reasoning capabilities of logic programs. We evaluate our method on various grid puzzles and dynamic puzzles involving actions, demonstrating near-perfect accuracy across all tasks. Our code and data are available at: https://github.com/naiqili/Logic-of-Thought.