mlsanityData to 7 Oct 2026

Large Language Models Are Neurosymbolic Reasoners

arXiv 2401.09334 ↗First posted 17 Jan 2024cs.CL cs.AIcited by 31

On the map: LLM reasoning → Reasoning for agents, planning and code

also in 1 more
Abstract (arXiv)

A wide range of real-world applications is characterized by their symbolic nature, necessitating a strong capability for symbolic reasoning. This paper investigates the potential application of Large Language Models (LLMs) as symbolic reasoners. We focus on text-based games, significant benchmarks for agents with natural language capabilities, particularly in symbolic tasks like math, map reading, sorting, and applying common sense in text-based worlds. To facilitate these agents, we propose an LLM agent designed to tackle symbolic challenges and achieve in-game objectives. We begin by initializing the LLM agent and informing it of its role. The agent then receives observations and a set of valid actions from the text-based games, along with a specific symbolic module. With these inputs, the LLM agent chooses an action and interacts with the game environments. Our experimental results demonstrate that our method significantly enhances the capability of LLMs as automated agents for symbolic reasoning, and our LLM agent is effective in text-based games involving symbolic tasks, achieving an average performance of 88% across all tasks.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Placed in Reasoning for agents, planning and code, one of 17 directions in LLM reasoning. Builds on 8 of its 49 references that are in the map. First posted on arXiv 17 Jan 2024.

Builds on

8 of 49 references.

References · 8

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  2. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  3. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  4. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  5. 2022Inner Monologue: Embodied Reasoning through Planning with Language Modelscited by 1,057
  6. 2023Foundation Models for Decision Making: Problems, Methods, and Opportunitiescited by 160
  7. 2018Counting to Explore and Generalize in Text-based Gamescited by 36
  8. 2022Large Language Models can Implement Policy Iterationcited by 31

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 9; 2026: 7.

31 papers.

Sort:

By area:

  1. 2025ATLaS: Agent Tuning via Learning Critical Stepscited by 29
  2. 2025Monte Carlo Planning with Large Language Model for Text-Based Game Agentscited by 22
  3. 2024MALMM: Multi-Agent Large Language Models for Zero-Shot Robotics Manipulationcited by 18
  4. 2024SymbolicAI: A framework for logic-based approaches combining generative models and solverscited by 15
  5. 2024Towards Rationality in Language and Multimodal Agents: A Surveycited by 15
  6. 2025NeSyC: A Neuro-symbolic Continual Learner For Complex Embodied Tasks In Open Domainscited by 15
  7. 2024Jailbreaking Large Language Models with Symbolic Mathematicscited by 14
  8. 2024Large Language Models are Zero-Shot Recognizers for Activities of Daily Livingcited by 13
  9. 2024The Sociolinguistic Foundations of Language Modelingcited by 13
  10. 2024GraphEval36K: Benchmarking Coding and Reasoning Capabilities of Large Language Models on Graph Datasetscited by 12
  11. 2025Incentivizing Strong Reasoning from Weak Supervisioncited by 8
  12. 2024Large Language Models are Interpretable Learnerscited by 7
19 more
  1. 2024Dspy-based Neural-Symbolic Pipeline to Enhance Spatial Reasoning in LLMscited by 6
  2. 2024Speak It Out: Solving Symbol-Related Problems with Symbol-to-Language Conversion for Language Modelscited by 4
  3. 2024RuAG: Learned-rule-augmented Generation for Large Language Modelscited by 4
  4. 2024Enhancing Conflict Resolution in Language Models via Abstract Argumentationcited by 3
  5. 2025SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligencecited by 3
  6. 2026Self-evolving LLM agents with in-distribution Optimizationcited by 3
  7. 2025Neuro-Symbolic Frameworks: Conceptual Characterization and Empirical Comparative Analysiscited by 2
  8. 2024A Survey on the Integration of Generative AI for Critical Thinking in Mobile Networkscited by 1
  9. 2024Parameter Choice and Neuro-Symbolic Approaches for Deep Domain-Invariant Learningcited by 1
  10. 2026Say Anything but This: When Tokenizer Betrays Reasoning in LLMscited by 1
  11. 2026MALLVI: A Multi-Agent Framework for Integrated Generalized Robotics Manipulationcited by 1
  12. 2024Unsupervised multiple choices question answering via universal corpuscited by 0
  13. 2025Model-Grounded Symbolic Artificial Intelligence Systems Learning and Reasoning with Model-Grounded Symbolic Artificial Intelligence Systemscited by 0
  14. 2025Benchmarking Foundation Models with Retrieval-Augmented Generation in Olympic-Level Physics Problem Solvingcited by 0
  15. 2025Enabling Fine-Grained Operating Points for Black-Box LLMscited by 0
  16. 2026ID-LoRA: Efficient Low-Rank Adaptation Inspired by Matrix Interpolative Decompositioncited by 0
  17. 2026Delta1 with LLM: symbolic and neural integration for credible and explainable reasoningcited by 0
  18. 2026Homoglyph-based Adversarial Perturbation of Introductory Computer Science Theory Problemscited by 0
  19. 2026SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videoscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Codenames as a Benchmark for Large Language Models
  2. 2024LogicGame: Benchmarking Rule-Based Reasoning Abilities of Large Language Models
  3. 2024Towards Logically Sound Natural Language Reasoning with Logic-Enhanced Language Model Agents
  4. 2026Reasoning Capabilities of Large Language Models. Lessons Learned from General Game Playing
  5. 2024Assessing the Emergent Symbolic Reasoning Abilities of Llama Large Language Models
  6. 2024Can Large Language Models Act as Symbolic Reasoners?
  7. 2025Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
  8. 2026Clueing up LLMs with Tool-Augmented Deductive Reasoning