# Logically Consistent Language Models via Neuro-Symbolic Integration

arXiv: https://arxiv.org/abs/2409.13724 · HTML: https://mlsanity.com/paper/2409.13724/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 75 references that are in the map.
- First posted on arXiv 9 Sep 2024.

## Builds on (most cited first, 30 of 58)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [A Prompt Pattern Catalog to Enhance Prompt Engineering with ChatGPT](https://mlsanity.com/paper/2302.11382/) (2023)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"](https://mlsanity.com/paper/2309.12288/) (2023)
- [A Survey of Hallucination in Large Foundation Models](https://mlsanity.com/paper/2309.05922/) (2023)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [A Semantic Loss Function for Deep Learning with Symbolic Knowledge](https://mlsanity.com/paper/1711.11157/) (2017)
- [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019)
- [Evaluating the Ripple Effects of Knowledge Editing in Language Models](https://mlsanity.com/paper/2307.12976/) (2023)
- [Differentiation of Blackbox Combinatorial Solvers](https://mlsanity.com/paper/1912.02175/) (2019)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://mlsanity.com/paper/2310.07521/) (2023)
- [Neural-Symbolic Learning and Reasoning: A Survey and Interpretation](https://mlsanity.com/paper/1711.03902/) (2017)
- [CLUTRR: A Diagnostic Benchmark for Inductive Reasoning from Text](https://mlsanity.com/paper/1908.06177/) (2019)
- [Maieutic Prompting: Logically Consistent Reasoning with Recursive Explanations](https://mlsanity.com/paper/2205.11822/) (2022)
- [A Review on Language Models as Knowledge Bases](https://mlsanity.com/paper/2204.06031/) (2022)
- [Factuality Challenges in the Era of Large Language Models](https://mlsanity.com/paper/2310.05189/) (2023)
- [Truthful AI: Developing and governing AI that does not lie](https://mlsanity.com/paper/2110.06674/) (2021)
- [Semantic Probabilistic Layers for Neuro-Symbolic Learning](https://mlsanity.com/paper/2206.00426/) (2022)
- [Analyzing Differentiable Fuzzy Logic Operators](https://mlsanity.com/paper/2002.06100/) (2020)
- [A Differential Approach to Inference in Bayesian Networks](https://mlsanity.com/paper/1301.3847/) (2013)
- [On the Paradox of Learning to Reason from Data](https://mlsanity.com/paper/2205.11502/) (2022)

## Built on by (most cited first, 30 of 30)

- [Scaling Reasoning Efficiently via Relaxed On-Policy Distillation](https://mlsanity.com/paper/2603.11137/) (2026)
- [Empowering LLMs with Logical Reasoning: A Comprehensive Survey](https://mlsanity.com/paper/2502.15652/) (2025)
- [Towards Robust Legal Reasoning: Harnessing Logical LLMs in Law](https://mlsanity.com/paper/2502.17638/) (2025)
- [Beyond the convexity assumption: Realistic tabular data generation under quantifier-free real linear constraints](https://mlsanity.com/paper/2502.18237/) (2025)
- [On Verifiable Legal Reasoning: A Multi-Agent Framework with Formalized Knowledge Representations](https://mlsanity.com/paper/2509.00710/) (2025)
- [SymRTLO: Enhancing RTL Code Optimization with LLMs and Neuron-Inspired Symbolic Reasoning](https://mlsanity.com/paper/2504.10369/) (2025)
- [If Concept Bottlenecks are the Question, are Foundation Models the Answer?](https://mlsanity.com/paper/2504.19774/) (2025)
- [Explainable Rule Application via Structured Prompting: A Neural-Symbolic Approach](https://mlsanity.com/paper/2506.16335/) (2025)
- [Neurosymbolic Diffusion Models](https://mlsanity.com/paper/2505.13138/) (2025)
- [Comprehension Without Competence: Architectural Limits of LLMs in Symbolic Computation and Reasoning](https://mlsanity.com/paper/2507.10624/) (2025)
- [Are Large Language Models Robust in Understanding Code Against Semantics-Preserving Mutations?](https://mlsanity.com/paper/2505.10443/) (2025)
- [Beyond Calibration: Do a Typed-Decision Model's Probabilities Obey the Probability Axioms?](https://mlsanity.com/paper/2609.33209/) (2026)
- [Understanding the Logic of Direct Preference Alignment through Logic](https://mlsanity.com/paper/2412.17696/) (2024)
- [To Neuro-Symbolic Classification and Beyond by Compiling Description Logic Ontologies to Probabilistic Circuits](https://mlsanity.com/paper/2601.14894/) (2026)
- [Barriers to Discrete Reasoning with Transformers: A Survey Across Depth, Exactness, and Bandwidth](https://mlsanity.com/paper/2602.11175/) (2026)
- [The Knowledge-Reasoning Dissociation: Fundamental Limitations of LLMs in Clinical Natural Language Inference](https://mlsanity.com/paper/2508.10777/) (2025)
- [Concepts Worth Having: Refining VLM-Guided Concept Bottleneck Models with Minimal Annotations](https://mlsanity.com/paper/2605.16405/) (2026)
- [Structured Decomposition for LLM Reasoning: Cross-Domain Validation and Semantic Web Integration](https://mlsanity.com/paper/2601.01609/) (2026)
- [Position: Logical Soundness is not a Reliable Criterion for Neurosymbolic Fact-Checking with LLMs](https://mlsanity.com/paper/2604.04177/) (2026)
- [Hierarchical Alignment: Enforcing Hierarchical Instruction-Following in LLMs through Logical Consistency](https://mlsanity.com/paper/2604.09075/) (2026)
- [From Token Probabilities to Semantic Constraints: Towards Declarative Probabilistic Evaluation of Language Models](https://mlsanity.com/paper/2609.13520/) (2026)
- [LogicTrack: Auditing Reasoning Trajectories of Large Language Models with Formal Logic Solvers](https://mlsanity.com/paper/2609.21492/) (2026)
- [Causal Reasoning Favors Encoders: On The Limits of Decoder-Only Models](https://mlsanity.com/paper/2512.10561/) (2025)
- [A Neuro-Symbolic Framework for Accountability in Public-Sector AI](https://mlsanity.com/paper/2512.12109/) (2025)
- [Explicit Logic Channel for Validation and Enhancement of MLLMs on Zero-Shot Tasks](https://mlsanity.com/paper/2603.11689/) (2026)
- [Can VLMs Reason Robustly? A Neuro-Symbolic Investigation](https://mlsanity.com/paper/2603.23867/) (2026)
- [Reliable Reasoning with Large Language Models via Preference-Based Maximum Satisfiability](https://mlsanity.com/paper/2605.29687/) (2026)
- [SymDiag: Explainable Diagnosis for LLM Reasoning via Neuro-Symbolic Verification](https://mlsanity.com/paper/2608.08786/) (2026)
- [Solving versus Verifying: Catching Contradictions in Tax Reasoning Systems](https://mlsanity.com/paper/2609.05928/) (2026)
- [Understanding as No-Arbitrage: Bounded Dutch Books as a Definition and Training Objective for Language Models](https://mlsanity.com/paper/2609.39341/) (2026)

## Abstract (arXiv)

Large language models (LLMs) are a promising venue for natural language understanding and generation. However, current LLMs are far from reliable: they are prone to generating non-factual information and, more crucially, to contradicting themselves when prompted to reason about relations between entities of the world. These problems are currently addressed with large scale fine-tuning or by delegating reasoning to external tools. In this work, we strive for a middle ground and introduce a loss based on neuro-symbolic reasoning that teaches an LLM to be logically consistent with an external set of facts and rules and improves self-consistency even when the LLM is fine-tuned on a limited set of facts. Our approach also allows to easily combine multiple logical constraints at once in a principled way, delivering LLMs that are more consistent w.r.t. all constraints and improve over several baselines w.r.t. a given constraint. Moreover, our method allows LLMs to extrapolate to unseen but semantically similar factual knowledge, represented in unseen datasets, more systematically.