# Stochastic LLMs do not Understand Language: Towards Symbolic, Explainable and Ontologically Based LLMs

arXiv: https://arxiv.org/abs/2309.05918 · HTML: https://mlsanity.com/paper/2309.05918/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 31 references that are in the map.
- First posted on arXiv 12 Sep 2023.

## Builds on (most cited first, 3 of 3)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Distributional Semantics and Linguistic Theory](https://mlsanity.com/paper/1905.01896/) (2019)
- [On Degrees of Freedom in Defining and Testing Natural Language Understanding](https://mlsanity.com/paper/2305.15130/) (2023)

## Built on by (most cited first, 9 of 9)

- [On Large Language Models in National Security Applications](https://mlsanity.com/paper/2407.03453/) (2024)
- [Can LLMs perform structured graph reasoning?](https://mlsanity.com/paper/2402.01805/) (2024)
- [Exploring LLM-Driven Explanations for Quantum Algorithms](https://mlsanity.com/paper/2409.19028/) (2024)
- [CogGPT: Unleashing the Power of Cognitive Dynamics on Large Language Models](https://mlsanity.com/paper/2401.08438/) (2024)
- [Generative Artificial Intelligence: Evolving Technology, Growing Societal Impact, and Opportunities for Information Systems Research](https://mlsanity.com/paper/2503.05770/) (2025)
- [Knowing the Facts but Choosing the Shortcut: Understanding How Large Language Models Compare Entities](https://mlsanity.com/paper/2510.16815/) (2025)
- [The Problem of Alignment](https://mlsanity.com/paper/2401.00210/) (2023)
- [Quantification and object perception in Multimodal Large Language Models and human linguistic cognition](https://mlsanity.com/paper/2511.08126/) (2025)
- [Representation Robustness Under Executable Reasoning Constraints in Large Language Models for Mathematical Problem Solving](https://mlsanity.com/paper/2607.20520/) (2026)

## Abstract (arXiv)

In our opinion the exuberance surrounding the relative success of data-driven large language models (LLMs) is slightly misguided and for several reasons (i) LLMs cannot be relied upon for factual information since for LLMs all ingested text (factual or non-factual) was created equal; (ii) due to their subsymbolic na-ture, whatever 'knowledge' these models acquire about language will always be buried in billions of microfeatures (weights), none of which is meaningful on its own; and (iii) LLMs will often fail to make the correct inferences in several linguistic contexts (e.g., nominal compounds, copredication, quantifier scope ambi-guities, intensional contexts. Since we believe the relative success of data-driven large language models (LLMs) is not a reflection on the symbolic vs. subsymbol-ic debate but a reflection on applying the successful strategy of a bottom-up reverse engineering of language at scale, we suggest in this paper applying the effective bottom-up strategy in a symbolic setting resulting in symbolic, explainable, and ontologically grounded language models.