# Inside-Out: Hidden Factual Knowledge in LLMs

arXiv: https://arxiv.org/abs/2503.15299 · HTML: https://mlsanity.com/paper/2503.15299/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Mar 2025.

## Built on by (most cited first, 7 of 7)

- [Temporal Predictors of Outcome in Reasoning Language Models](https://mlsanity.com/paper/2511.14773/) (2025)
- [Efficient Reasoning with Balanced Thinking](https://mlsanity.com/paper/2603.12372/) (2026)
- [Higher-Order Knowledge Representations for Agentic Scientific Reasoning](https://mlsanity.com/paper/2601.04878/) (2026)
- [Input-Time Scaling: Adding Noise and Irrelevance into Less-Is-More Drastically Improves Reasoning Performance and Efficiency](https://mlsanity.com/paper/2508.13654/) (2025)
- [Knowing but Not Saying: Preventing Factual Access Failures in LLM SFT via Recall-Anchored Distillation](https://mlsanity.com/paper/2608.20794/) (2026)
- [Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Models](https://mlsanity.com/paper/2609.33290/) (2026)
- [Structured but Silent: Probing Capability Requirements in LLM Hidden States](https://mlsanity.com/paper/2610.08018/) (2026)

## Abstract (arXiv)

This work presents a framework for assessing whether large language models (LLMs) encode more factual knowledge in their parameters than what they express in their outputs. While a few studies hint at this possibility, none has clearly defined or demonstrated this phenomenon. We first propose a formal definition of knowledge, quantifying it for a given question as the fraction of correct-incorrect answer pairs where the correct one is ranked higher. This gives rise to external and internal knowledge, depending on the information used to score individual answer candidates: either the model's observable token-level probabilities or its intermediate computations. Hidden knowledge arises when internal knowledge exceeds external knowledge. We then present a case study, applying this framework to three popular open-weights LLMs in a closed-book QA setup. Our results indicate that: (1) LLMs consistently encode more factual knowledge internally than what they express externally, with an average relative gap of 40%. (2) Surprisingly, some knowledge is so deeply hidden that a model can internally know an answer perfectly, yet fail to generate it even once, despite large-scale repeated sampling of 1,000 answers. This reveals fundamental limitations in the generation capabilities of LLMs, which (3) put a practical constraint on scaling test-time compute via repeated answer sampling in closed-book QA: significant performance improvements remain inaccessible because some answers are practically never sampled, yet if they were, we would be guaranteed to rank them first.