PrologMCP: A Standardized Prolog Tool Interface for LLM Agents
On the map: LLM reasoning → Reasoning benchmarks and evaluation
also in 1 more
Abstract (arXiv)
Frontier reasoning-tuned language models still fail on deductive tasks at depth, and the cost of improved performance through extended internal reasoning scales poorly. Symbolic delegation offers a complementary route: a language model translates the problem, while a solver performs the inference. However, current autoformalization pipelines for logic programming are typically bespoke integrations tied to particular tasks or agents. We introduce PrologMCP, a task-agnostic, open-source server that exposes Prolog as a stateful tool through the Model Context Protocol (MCP). Its compact tool interface, structured error reporting, and per-session isolation make the translate-run-inspect-repair loop a reusable primitive for MCP-capable agents. We evaluate a formalizer agent enhanced with PrologMCP against standard and reasoning LLMs (Claude Sonnet 4.6, GPT-4.1, and o4-mini) on two subsets of PARARULE-Plus: a general-purpose sample and a more challenging one targeting a specific failure mode of natural-language reasoning. On the general sample, the formalizer matches or exceeds reasoning LLMs (accuracy 1.00 vs. 1.00 / 0.998), with the largest gains over standard models (0.762 for GPT-4.1). On the challenging subset, the formalizer remains near-perfect (1.00 / 0.99) while reasoning LLMs drop to 0.95 / 0.94. These results suggest that delegating inference to Prolog via MCP is a robust and inspectable alternative to extended natural-language reasoning.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 18 of its 31 references that are in the map. First posted on arXiv 12 Jun 2026.
Builds on
18 of 31 references.
Closest ancestors · 6
Same direction, newest first
- 2026NeuroProlog: Multi-Task Fine-Tuning for Neurosymbolic Mathematical Reasoning via the Cocktail Effect3 Mar
- 2025ProtoReasoning: Prototypes as the Foundation for Generalizable Reasoning in LLMs18 Jun
- 2025BIG-Bench Extra Hard26 Feb
- 2025ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning3 Feb
- 2024Reliable Reasoning Beyond Natural Language16 Jul
- 2022Multi-Step Deductive Reasoning Over Natural Language: An Empirical Study on Out-of-Distribution Generalisation28 Jul
Other references · 12
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 533
- 2023Faith and Fate: Limits of Transformers on Compositionalitycited by 496
- 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexitycited by 298
- 2020Transformers as Soft Reasoners over Languagecited by 265
- 2025Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directionscited by 257
- 2024Arithmetic Reasoning with LLM: Prolog Generation & Permutationcited by 17
- 2022Fifty Years of Prolog and Beyondcited by 15
- 2025Towards a Common Framework for Autoformalizationcited by 7
- 2024Generative Agents for Multi-Agent Autoformalization of Interaction Scenarioscited by 5
- 2024Thought-Like-Pro: Enhancing Reasoning of Large Language Models through Self-Driven Prolog-based Chain-of-Thoughtcited by 3
- 2025Training Language Models to Use Prolog as a Toolcited by 1
Built on by
Citing papers per year: 2026: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2024Towards Logically Sound Natural Language Reasoning with Logic-Enhanced Language Model Agents
- 2026A Comprehensive Evaluation of LLM Reasoning: From Single-Model to Multi-Agent Paradigms
- 2024ChatLogic: Integrating Logic Programming with Large Language Models for Multi-Step Reasoning
- 2026ASP-Bench: From Natural Language to Logic Programs
- 2025Reasoning on a Budget: A Survey of Adaptive and Controllable Test-Time Compute in LLMs
- 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systems
- 2026Pushing the Boundaries of Natural Reasoning: Interleaved Bonus from Formal-Logic Verification
- 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning