# Limits for Learning with Language Models

arXiv: https://arxiv.org/abs/2306.12213 · HTML: https://mlsanity.com/paper/2306.12213/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 59 references that are in the map.
- First posted on arXiv 21 Jun 2023.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [CoQA: A Conversational Question Answering Challenge](https://mlsanity.com/paper/1808.07042/) (2018)
- [Exploring Generalization in Deep Learning](https://mlsanity.com/paper/1706.08947/) (2017)
- [What do you learn from context? Probing for sentence structure in contextualized word representations](https://mlsanity.com/paper/1905.06316/) (2019)
- [Augmented Language Models: a Survey](https://mlsanity.com/paper/2302.07842/) (2023)
- [Physics Informed Deep Learning (Part I): Data-driven Solutions of Nonlinear Partial Differential Equations](https://mlsanity.com/paper/1711.10561/) (2017)
- [Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Fly](https://mlsanity.com/paper/1911.03343/) (2019)
- [Stress Test Evaluation for Natural Language Inference](https://mlsanity.com/paper/1806.00692/) (2018)
- [Generalization in Deep Learning](https://mlsanity.com/paper/1710.05468/) (2017)
- [On the Practical Computational Power of Finite Precision RNNs for Language Recognition](https://mlsanity.com/paper/1805.04908/) (2018)
- [Understanding by Understanding Not: Modeling Negation in Language Models](https://mlsanity.com/paper/2105.03519/) (2021)
- [UnNatural Language Inference](https://mlsanity.com/paper/2101.00010/) (2020)
- [Can stable and accurate neural networks be computed? -- On the barriers of deep learning and Smale's 18th problem](https://mlsanity.com/paper/2101.08286/) (2021)
- [Negation, Coordination, and Quantifiers in Contextualized Language Models](https://mlsanity.com/paper/2209.07836/) (2022)
- [On Learnability, Complexity and Stability](https://mlsanity.com/paper/1303.5976/) (2013)
- [The meaning of "most" for visual question answering models](https://mlsanity.com/paper/1812.11737/) (2018)

## Built on by (most cited first, 17 of 17)

- [Challenges and Applications of Large Language Models](https://mlsanity.com/paper/2307.10169/) (2023)
- [AirGapAgent: Protecting Privacy-Conscious Conversational Agents](https://mlsanity.com/paper/2405.05175/) (2024)
- [Large Language Models for Compiler Optimization](https://mlsanity.com/paper/2309.07062/) (2023)
- [Reflect, Retry, Reward: Self-Improving LLMs via Reinforcement Learning](https://mlsanity.com/paper/2505.24726/) (2025)
- [Easy Problems That LLMs Get Wrong](https://mlsanity.com/paper/2405.19616/) (2024)
- [Token embeddings violate the manifold hypothesis](https://mlsanity.com/paper/2504.01002/) (2025)
- [Machine-assisted quantitizing designs: augmenting humanities and social sciences with artificial intelligence](https://mlsanity.com/paper/2309.14379/) (2023)
- [Enhancing Post-Hoc Attributions in Long Document Comprehension via Coarse Grained Answer Decomposition](https://mlsanity.com/paper/2409.17073/) (2024)
- [Exploring Failure Cases in Multimodal Reasoning About Physical Dynamics](https://mlsanity.com/paper/2402.15654/) (2024)
- [It Ain't That Bad: Understanding the Mysterious Performance Drop in OOD Generalization for Generative Transformer Models](https://mlsanity.com/paper/2308.08268/) (2023)
- [Fine-tuning and Prompt Engineering with Cognitive Knowledge Graphs for Scholarly Knowledge Organization](https://mlsanity.com/paper/2409.06433/) (2024)
- [Rosetta-PL: Propositional Logic as a Benchmark for Large Language Model Reasoning](https://mlsanity.com/paper/2505.00001/) (2025)
- [Should AI Optimize Your Code? A Comparative Study of Classical Optimizing Compilers Versus Current Large Language Models](https://mlsanity.com/paper/2406.12146/) (2024)
- [The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models](https://mlsanity.com/paper/2601.19926/) (2026)
- [Multilingual Large Language Models do not comprehend all natural languages to equal degrees](https://mlsanity.com/paper/2602.20065/) (2026)
- [Trustworthy Formal Natural Language Specifications](https://mlsanity.com/paper/2310.03885/) (2023)
- [LLM-Augmented Symbolic NLU System for More Reliable Continuous Causal Statement Interpretation](https://mlsanity.com/paper/2510.19988/) (2025)

## Abstract (arXiv)

With the advent of large language models (LLMs), the trend in NLP has been to train LLMs on vast amounts of data to solve diverse language understanding and generation tasks. The list of LLM successes is long and varied. Nevertheless, several recent papers provide empirical evidence that LLMs fail to capture important aspects of linguistic meaning. Focusing on universal quantification, we provide a theoretical foundation for these empirical findings by proving that LLMs cannot learn certain fundamental semantic properties including semantic entailment and consistency as they are defined in formal semantics. More generally, we show that LLMs are unable to learn concepts beyond the first level of the Borel Hierarchy, which imposes severe limits on the ability of LMs, both large and small, to capture many aspects of linguistic meaning. This means that LLMs will continue to operate without formal guarantees on tasks that require entailments and deep linguistic understanding.