# Extending Machine Language Models toward Human-Level Language Understanding

arXiv: https://arxiv.org/abs/1912.05877 · HTML: https://mlsanity.com/paper/1912.05877/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Dec 2019.

## Built on by (most cited first, 7 of 7)

- [Cognitive Architectures for Language Agents](https://mlsanity.com/paper/2309.02427/) (2023)
- [PIGLeT: Language Grounding Through Neuro-Symbolic Interaction in a 3D World](https://mlsanity.com/paper/2106.00188/) (2021)
- [Creating Multimodal Interactive Agents with Imitation and Self-Supervised Learning](https://mlsanity.com/paper/2112.03763/) (2021)
- [UNVEILING: What Makes Linguistics Olympiad Puzzles Tricky for LLMs?](https://mlsanity.com/paper/2508.11260/) (2025)
- [Shades of meaning: Uncovering the geometry of ambiguous word representations through contextualised language models](https://mlsanity.com/paper/2304.13597/) (2023)
- [Polishing Every Facet of the GEM: Testing Linguistic Competence of LLMs and Humans in Korean](https://mlsanity.com/paper/2506.01237/) (2025)
- [Generalized Optimal Linear Orders](https://mlsanity.com/paper/2108.10692/) (2021)

## Abstract (arXiv)

Language is crucial for human intelligence, but what exactly is its role? We take language to be a part of a system for understanding and communicating about situations. The human ability to understand and communicate about situations emerges gradually from experience and depends on domain-general principles of biological neural networks: connection-based learning, distributed representation, and context-sensitive, mutual constraint satisfaction-based processing. Current artificial language processing systems rely on the same domain general principles, embodied in artificial neural networks. Indeed, recent progress in this field depends on query-based attention, which extends the ability of these systems to exploit context and has contributed to remarkable breakthroughs. Nevertheless, most current models focus exclusively on language-internal tasks, limiting their ability to perform tasks that depend on understanding situations. These systems also lack memory for the contents of prior situations outside of a fixed contextual span. We describe the organization of the brain's distributed understanding system, which includes a fast learning system that addresses the memory problem. We sketch a framework for future models of understanding drawing equally on cognitive neuroscience and artificial intelligence and exploiting query-based attention. We highlight relevant current directions and consider further developments needed to fully capture human-level language understanding in a computational system.