# Mirage of Mastery: Memorization Tricks LLMs into Artificially Inflated Self-Knowledge

arXiv: https://arxiv.org/abs/2506.18998 · HTML: https://mlsanity.com/paper/2506.18998/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 33 references that are in the map.
- First posted on arXiv 23 Jun 2025.

## Builds on (most cited first, 27 of 27)

- [GPT-4o System Card](https://mlsanity.com/paper/2410.21276/) (2024)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [RouteLLM: Learning to Route LLMs with Preference Data](https://mlsanity.com/paper/2406.18665/) (2024)
- [Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentation](https://mlsanity.com/paper/2307.11019/) (2023)
- [On Memorization of Large Language Models in Logical Reasoning](https://mlsanity.com/paper/2410.23123/) (2024)
- [Can Large Language Models Faithfully Express Their Intrinsic Uncertainty in Words?](https://mlsanity.com/paper/2405.16908/) (2024)
- [MEDEC: A Benchmark for Medical Error Detection and Correction in Clinical Notes](https://mlsanity.com/paper/2412.19260/) (2024)
- [Self-Knowledge Guided Retrieval Augmentation for Large Language Models](https://mlsanity.com/paper/2310.05002/) (2023)
- [When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentation](https://mlsanity.com/paper/2402.11457/) (2024)
- [Demystifying Verbatim Memorization in Large Language Models](https://mlsanity.com/paper/2407.17817/) (2024)
- [Exploring Memorization in Fine-tuned Language Models](https://mlsanity.com/paper/2310.06714/) (2023)
- [Benchmarking and Improving Generator-Validator Consistency of Language Models](https://mlsanity.com/paper/2310.01846/) (2023)
- [SoK: Memorization in General-Purpose Large Language Models](https://mlsanity.com/paper/2310.18362/) (2023)
- [Elephants Never Forget: Memorization and Learning of Tabular Data in Large Language Models](https://mlsanity.com/paper/2404.06209/) (2024)
- [Recite, Reconstruct, Recollect: Memorization in LMs as a Multifaceted Phenomenon](https://mlsanity.com/paper/2406.17746/) (2024)
- [Undesirable Memorization in Large Language Models: A Survey](https://mlsanity.com/paper/2410.02650/) (2024)
- [LLM-Generated Black-box Explanations Can Be Adversarially Helpful](https://mlsanity.com/paper/2405.06800/) (2024)
- [Unlocking Memorization in Large Language Models with Dynamic Soft Prompting](https://mlsanity.com/paper/2409.13853/) (2024)
- [Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answering](https://mlsanity.com/paper/2405.14383/) (2024)
- [Self-Recognition in Language Models](https://mlsanity.com/paper/2407.06946/) (2024)
- [Self-Cognition in Large Language Models: An Exploratory Study](https://mlsanity.com/paper/2407.01505/) (2024)
- [Question-Analysis Prompting Improves LLM Performance in Reasoning Tasks](https://mlsanity.com/paper/2407.03624/) (2024)
- [Understanding Memorisation in LLMs: Dynamics, Influencing Factors, and Implications](https://mlsanity.com/paper/2407.19262/) (2024)
- [Line of Duty: Evaluating LLM Self-Knowledge via Consistency in Feasibility Boundaries](https://mlsanity.com/paper/2503.11256/) (2025)
- [Can I understand what I create? Self-Knowledge Evaluation of Large Language Models](https://mlsanity.com/paper/2406.06140/) (2024)
- [Detecting Memorization in Large Language Models](https://mlsanity.com/paper/2412.01014/) (2024)
- [Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models](https://mlsanity.com/paper/2503.01208/) (2025)

## Built on by (most cited first, 2 of 2)

- [KnowRL: Teaching Language Models to Know What They Know](https://mlsanity.com/paper/2510.11407/) (2025)
- [Look It Up: Analysing Internal Web Search Capabilities of Modern LLMs](https://mlsanity.com/paper/2511.18931/) (2025)

## Abstract (arXiv)

When artificial intelligence mistakes memorization for intelligence, it creates a dangerous mirage of reasoning. Existing studies treat memorization and self-knowledge deficits in LLMs as separate issues and do not recognize an intertwining link that degrades the trustworthiness of LLM responses. In our study, we utilize a novel framework to ascertain if LLMs genuinely learn reasoning patterns from training data or merely memorize them to assume competence across problems of similar complexity focused on STEM domains. Our analysis shows a noteworthy problem in generalization: LLMs draw confidence from memorized solutions to infer a higher self-knowledge about their reasoning ability, which manifests as an over 45% inconsistency in feasibility assessments when faced with self-validated, logically coherent task perturbations. This effect is most pronounced in science and medicine domains, which tend to have maximal standardized jargon and problems, further confirming our approach. Significant wavering within the self-knowledge of LLMs also shows flaws in current architectures and training patterns, highlighting the need for techniques that ensure a balanced, consistent stance on models' perceptions of their own knowledge for maximum AI explainability and trustworthiness. Our code and results are available publicly at https://github.com/Sahil-R-Kale/mirage_of_mastery