# `Generalization is hallucination' through the lens of tensor completions

arXiv: https://arxiv.org/abs/2502.17305 · HTML: https://mlsanity.com/paper/2502.17305/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 20 references that are in the map.
- First posted on arXiv 24 Feb 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [Intrinsic Dimensionality Explains the Effectiveness of Language Model Fine-Tuning](https://mlsanity.com/paper/2012.13255/) (2020)
- [Hallucination is Inevitable: An Innate Limitation of Large Language Models](https://mlsanity.com/paper/2401.11817/) (2024)
- [Stronger generalization bounds for deep nets via a compression approach](https://mlsanity.com/paper/1802.05296/) (2018)
- [LLMs Will Always Hallucinate, and We Need to Live With This](https://mlsanity.com/paper/2409.05746/) (2024)
- [PAC-Bayes Compression Bounds So Tight That They Can Explain Generalization](https://mlsanity.com/paper/2211.13609/) (2022)
- [Non-Vacuous Generalization Bounds for Large Language Models](https://mlsanity.com/paper/2312.17173/) (2023)
- [Understanding Factual Recall in Transformers via Associative Memories](https://mlsanity.com/paper/2412.06538/) (2024)
- [Tensor Completion Algorithms in Big Data Analytics](https://mlsanity.com/paper/1711.10105/) (2017)
- [Confabulation: The Surprising Value of Large Language Model Hallucinations](https://mlsanity.com/paper/2406.04175/) (2024)
- [Scaling Laws for Fact Memorization of Large Language Models](https://mlsanity.com/paper/2406.15720/) (2024)
- [I Don't Know: Explicit Modeling of Uncertainty with an [IDK] Token](https://mlsanity.com/paper/2412.06676/) (2024)
- [Banishing LLM Hallucinations Requires Rethinking Generalization](https://mlsanity.com/paper/2406.17642/) (2024)

## Abstract (arXiv)

In this short position paper, we introduce tensor completions and artifacts and make the case that they are a useful theoretical framework for understanding certain types of hallucinations and generalizations in language models.