# Knowledge Boundary of Large Language Models: A Survey

arXiv: https://arxiv.org/abs/2412.12472 · HTML: https://mlsanity.com/paper/2412.12472/

## Key facts

- Cited by 40 papers in the map (data through 14 Oct 2026).
- Builds on 141 of its 181 references that are in the map.
- First posted on arXiv 17 Dec 2024.

## Builds on (most cited first, 30 of 141)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [Language Models (Mostly) Know What They Know](https://mlsanity.com/paper/2207.05221/) (2022)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [Improving Factuality and Reasoning in Language Models through Multiagent Debate](https://mlsanity.com/paper/2305.14325/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Steps](https://mlsanity.com/paper/2011.01060/) (2020)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://mlsanity.com/paper/2104.08786/) (2021)
- [When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories](https://mlsanity.com/paper/2212.10511/) (2022)
- [Measuring and Narrowing the Compositionality Gap in Language Models](https://mlsanity.com/paper/2210.03350/) (2022)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [Large Language Models Cannot Self-Correct Reasoning Yet](https://mlsanity.com/paper/2310.01798/) (2023)
- [Mass-Editing Memory in a Transformer](https://mlsanity.com/paper/2210.07229/) (2022)
- [Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMs](https://mlsanity.com/paper/2306.13063/) (2023)
- [Large Language Models as Optimizers](https://mlsanity.com/paper/2309.03409/) (2023)
- [Large Language Models Can Be Easily Distracted by Irrelevant Context](https://mlsanity.com/paper/2302.00093/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [REPLUG: Retrieval-Augmented Black-Box Language Models](https://mlsanity.com/paper/2301.12652/) (2023)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Teaching Models to Express Their Uncertainty in Words](https://mlsanity.com/paper/2205.14334/) (2022)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [Large Language Models Can Self-Improve](https://mlsanity.com/paper/2210.11610/) (2022)

## Built on by (most cited first, 30 of 40)

- [SMART: Self-Aware Agent for Tool Overuse Mitigation](https://mlsanity.com/paper/2502.11435/) (2025)
- [CacheClip: Accelerating RAG with Effective KV Cache Reuse](https://mlsanity.com/paper/2510.10129/) (2025)
- [Query-Level Uncertainty in Large Language Models](https://mlsanity.com/paper/2506.09669/) (2025)
- [Enhancing LLM Reliability via Explicit Knowledge Boundary Modeling](https://mlsanity.com/paper/2503.02233/) (2025)
- [A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directions](https://mlsanity.com/paper/2504.16939/) (2025)
- [BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs](https://mlsanity.com/paper/2505.13529/) (2025)
- [INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profiling](https://mlsanity.com/paper/2505.16303/) (2025)
- [Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoning](https://mlsanity.com/paper/2505.20664/) (2025)
- [Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHub](https://mlsanity.com/paper/2601.00477/) (2026)
- [SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2601.03979/) (2026)
- [Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representations](https://mlsanity.com/paper/2504.13816/) (2025)
- [Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learning](https://mlsanity.com/paper/2510.02324/) (2025)
- [Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Risks](https://mlsanity.com/paper/2505.19806/) (2025)
- [Do Retrieval Augmented Language Models Know When They Don't Know?](https://mlsanity.com/paper/2509.01476/) (2025)
- [Understanding the Dilemma of Unlearning for Large Language Models](https://mlsanity.com/paper/2509.24675/) (2025)
- [Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Models](https://mlsanity.com/paper/2509.24711/) (2025)
- [STRUCTSURVEY: Structured Agentic Retrieval for Automated Survey Paper Generation](https://mlsanity.com/paper/2607.01243/) (2026)
- [Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Models](https://mlsanity.com/paper/2602.12996/) (2026)
- [Architecting Trust in Artificial Epistemic Agents](https://mlsanity.com/paper/2603.02960/) (2026)
- [KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditing](https://mlsanity.com/paper/2605.29524/) (2026)
- [Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detection](https://mlsanity.com/paper/2511.21066/) (2025)
- [Neurosymbolic Learning for Inference-Time Argumentation](https://mlsanity.com/paper/2605.20098/) (2026)
- [LORE: A Large Generative Model for Search Relevance](https://mlsanity.com/paper/2512.03025/) (2025)
- [GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamics](https://mlsanity.com/paper/2604.02830/) (2026)
- [What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systems](https://mlsanity.com/paper/2605.20548/) (2026)
- [Localizing RL-Induced Tool Use to a Single Crosscoder Feature](https://mlsanity.com/paper/2606.26474/) (2026)
- [ReFactX: Scalable Reasoning with Reliable Facts via Constrained Generation](https://mlsanity.com/paper/2508.16983/) (2025)
- [Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Reranking](https://mlsanity.com/paper/2509.06472/) (2025)
- [Assessing the Software Security Comprehension of Large Language Models](https://mlsanity.com/paper/2512.21238/) (2025)
- [Toward IIT-Inspired Consciousness in LLMs: A Reward-Based Learning Framework](https://mlsanity.com/paper/2601.22786/) (2026)

## Abstract (arXiv)

Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses. This highlights the critical need to understand the knowledge boundary of LLMs, a concept that remains inadequately defined in existing research. In this survey, we propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. Using this foundation, we systematically review the field through three key lenses: the motivation for studying LLM knowledge boundaries, methods for identifying these boundaries, and strategies for mitigating the challenges they present. Finally, we discuss open challenges and potential research directions in this area. We aim for this survey to offer the community a comprehensive overview, facilitate access to key issues, and inspire further advancements in LLM knowledge research.