Intermediate Languages Matter: Formal Languages and LLMs affect Neurosymbolic Reasoning
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Large language models (LLMs) achieve astonishing results on a wide range of tasks. However, their formal reasoning ability still lags behind. A promising approach is Neurosymbolic LLM reasoning. It works by using LLMs as translators from natural to formal languages and symbolic solvers for deriving correct results. Still, the contributing factors to the success of Neurosymbolic LLM reasoning remain unclear. This paper demonstrates that one previously overlooked factor is the choice of the formal language. We introduce the intermediate language challenge: selecting a suitable formal language for neurosymbolic reasoning. By comparing four formal languages across three datasets and seven LLMs, we show that the choice of formal language affects both syntactic and semantic reasoning capabilities. We also discuss the varying effects across different LLMs.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 20 of its 30 references that are in the map. First posted on arXiv 4 Sep 2025.
Builds on
20 of 30 references.
Closest ancestors · 7
Same direction, newest first
- 2025Logic-of-Thought: Empowering Large Language Models with Logic Programs for Solving Puzzles in Natural Language22 May
- 2025Intermediate Languages Matter: Formal Choice Drives Neurosymbolic LLM Reasoning24 Feb
- 2024LOGIC-LM++: Multi-Step Refinement for Symbolic Formulations22 Jun
- 2023LINC: A Neurosymbolic Approach for Logical Reasoning by Combining Language Models with First-Order Logic Provers23 Oct
- 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoning20 May
- 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thought3 Oct
- 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep
Other references · 13
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2020ProofWriter: Generating Implications, Proofs, and Abductive Statements over Natural Languagecited by 309
- 2023Faithful Chain-of-Thought Reasoningcited by 285
- 2020Neurosymbolic AI: The 3rd Wavecited by 253
- 2020Logic Tensor Networkscited by 197
- 2022Talking About Large Language Modelscited by 174
- 2020Injecting Numerical Reasoning Skills into Language Modelscited by 133
- 2020Birds have four legs?! NumerSense: Probing Numerical Commonsense Knowledge of Pre-trained Language Modelscited by 110
- 2023Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Modelscited by 92
- 2024LLASP: Fine-tuning Large Language Models for Answer Set Programmingcited by 8
- 2021On the Foundations of Grounding in Answer Set Programmingcited by 6
- 2024How Proficient Are Large Language Models in Formal Languages? An In-Depth Insight for Knowledge Base Question Answeringcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025Neuro-Symbolic Artificial Intelligence: Towards Improving the Reasoning Abilities of Large Language Models
- 2025Improving Rule-based Reasoning in LLMs using Neurosymbolic Representations
- 2024Reliable Reasoning Beyond Natural Language
- 2024Can Large Language Models Act as Symbolic Reasoners?
- 2025Current Practices for Building LLM-Powered Reasoning Tools Are Ad Hoc -- and We Can Do Better
- 2022Towards Reasoning in Large Language Models: A Survey
- 2025Robustness of Neurosymbolic Reasoners on First-Order Logic Problems
- 2025Do Large Language Models Excel in Complex Logical Reasoning with Formal Language?