# How LLMs Might Think

arXiv: https://arxiv.org/abs/2604.09674 · HTML: https://mlsanity.com/paper/2604.09674/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 57 references that are in the map.
- First posted on arXiv 2 Apr 2026.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [Probing Classifiers: Promises, Shortcomings, and Advances](https://mlsanity.com/paper/2102.12452/) (2021)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [What learning algorithm is in-context learning? Investigations with linear models](https://mlsanity.com/paper/2211.15661/) (2022)
- [Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task](https://mlsanity.com/paper/2210.13382/) (2022)
- [Structured Attention Networks](https://mlsanity.com/paper/1702.00887/) (2017)
- [Reuse, Don't Retrain: A Recipe for Continued Pretraining of Language Models](https://mlsanity.com/paper/2407.07263/) (2024)
- [From task structures to world models: What do LLMs know?](https://mlsanity.com/paper/2310.04276/) (2023)
- [Standards for Belief Representations in LLMs](https://mlsanity.com/paper/2405.21030/) (2024)
- [A Philosophical Introduction to Language Models -- Part I: Continuity With Classic Debates](https://mlsanity.com/paper/2401.03910/) (2024)
- [Model Surgery: Modulating LLM's Behavior Via Simple Parameter Editing](https://mlsanity.com/paper/2407.08770/) (2024)
- [Does ChatGPT Have a Mind?](https://mlsanity.com/paper/2407.11015/) (2024)
- [Propositional Interpretability in Artificial Intelligence](https://mlsanity.com/paper/2501.15740/) (2025)
- [Optimizing Large Language Model Hyperparameters for Code Generation](https://mlsanity.com/paper/2408.10577/) (2024)
- [Real Sparks of Artificial Intelligence and the Importance of Inner Interpretability](https://mlsanity.com/paper/2402.00901/) (2024)
- [Do Transformers know symbolic rules, and would we know if they did?](https://mlsanity.com/paper/2203.00162/) (2022)

## Abstract (arXiv)

Do large language models (LLMs) think? Daniel Stoljar and Zhihe Vincent Zhang have recently developed an argument from rationality for the claim that LLMs do not think. We contend, however, that the argument from rationality not only falters, but leaves open an intriguing possibility: that LLMs engage only in arational, associative forms of thinking, and have purely associative minds. Our positive claim is that if LLMs think at all, they likely think precisely in this manner.