mlsanityData to 13 Oct 2026

Knowledge Boundary of Large Language Models: A Survey

arXiv 2412.12472 ↗First posted 17 Dec 2024cs.CLcited by 38

Not on the map. Why?

Abstract (arXiv)

Although large language models (LLMs) store vast amount of knowledge in their parameters, they still have limitations in the memorization and utilization of certain knowledge, leading to undesired behaviors such as generating untruthful and inaccurate responses. This highlights the critical need to understand the knowledge boundary of LLMs, a concept that remains inadequately defined in existing research. In this survey, we propose a comprehensive definition of the LLM knowledge boundary and introduce a formalized taxonomy categorizing knowledge into four distinct types. Using this foundation, we systematically review the field through three key lenses: the motivation for studying LLM knowledge boundaries, methods for identifying these boundaries, and strategies for mitigating the challenges they present. Finally, we discuss open challenges and potential research directions in this area. We aim for this survey to offer the community a comprehensive overview, facilitate access to key issues, and inspire further advancements in LLM knowledge research.

Key facts. Cited by 38 papers in the map (data through 13 Oct 2026). Builds on 141 of its 181 references that are in the map. First posted on arXiv 17 Dec 2024.

Builds on

141 of 181 references.

References · 141

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,979
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,657
  3. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,716
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,367
  5. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,949
  6. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,925
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,650
  8. 2022Language Models (Mostly) Know What They Knowcited by 1,823
  9. 2023Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflectioncited by 1,815
  10. 2023Improving Factuality and Reasoning in Language Models through Multiagent Debatecited by 1,758
  11. 2019PubMedQA: A Dataset for Biomedical Research Question Answeringcited by 1,281
  12. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,280
38 more
  1. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,268
  2. 2020Constructing A Multi-hop QA Dataset for Comprehensive Evaluation of Reasoning Stepscited by 1,266
  3. 2021What Makes Good In-Context Examples for GPT-3?cited by 1,191
  4. 2021Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivitycited by 1,126
  5. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,111
  6. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,063
  7. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,031
  8. 2022Mass-Editing Memory in a Transformercited by 990
  9. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 983
  10. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 960
  11. 2023Large Language Models as Optimizerscited by 842
  12. 2023Large Language Models Can Be Easily Distracted by Irrelevant Contextcited by 829
  13. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 800
  14. 2023REPLUG: Retrieval-Augmented Black-Box Language Modelscited by 790
  15. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 750
  16. 2022Teaching Models to Express Their Uncertainty in Wordscited by 740
  17. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 693
  18. 2022Large Language Models Can Self-Improvecited by 659
  19. 2021Learning To Retrieve Prompts for In-Context Learningcited by 643
  20. 2023Automatic Prompt Optimization with "Gradient Descent" and Beam Searchcited by 637
  21. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 600
  22. 2021FinQA: A Dataset of Numerical Reasoning over Financial Datacited by 553
  23. 2022Precise Zero-Shot Dense Retrieval without Relevance Labelscited by 510
  24. 2022Memory-Based Model Editing at Scalecited by 446
  25. 2021TAT-QA: A Question Answering Benchmark on a Hybrid of Tabular and Textual Content in Financecited by 405
  26. 2021Measuring and Improving Consistency in Pretrained Language Modelscited by 363
  27. 2022RLPrompt: Optimizing Discrete Text Prompts with Reinforcement Learningcited by 363
  28. 2023Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergycited by 344
  29. 2022Language Models Are Greedy Reasoners: A Systematic Formal Analysis of Chain-of-Thoughtcited by 335
  30. 2024Benchmarking Retrieval-Augmented Generation for Medicinecited by 328
  31. 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 312
  32. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 305
  33. 2023MQuAKE: Assessing Knowledge Editing in Language Models via Multi-Hop Questionscited by 297
  34. 2023Can We Edit Factual Knowledge by In-Context Learning?cited by 285
  35. 2022Aging with GRACE: Lifelong Model Editing with Discrete Key-Value Adaptorscited by 270
  36. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 266
  37. 2022Prompting GPT-3 To Be Reliablecited by 259
  38. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 255

50 most cited of 141 All 141 (JSON)

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 19; 2026: 19.

38 papers.

Sort:

By area:

  1. 2025SMART: Self-Aware Agent for Tool Overuse Mitigationcited by 52
  2. 2025CacheClip: Accelerating RAG with Effective KV Cache Reusecited by 15
  3. 2025Query-Level Uncertainty in Large Language Modelscited by 14
  4. 2025Enhancing LLM Reliability via Explicit Knowledge Boundary Modelingcited by 13
  5. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  6. 2025BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMscited by 8
  7. 2025INFERENCEDYNAMICS: Efficient Routing Across LLMs through Structured Capability and Knowledge Profilingcited by 8
  8. 2025Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoningcited by 8
  9. 2026Security in the Age of AI Teammates: An Empirical Study of Agentic Pull Requests on GitHubcited by 7
  10. 2026SoK: Privacy Risks and Mitigations in Retrieval-Augmented Generation Systemscited by 7
  11. 2025Analyzing LLMs' Knowledge Boundary Cognition Across Languages Through the Lens of Internal Representationscited by 6
  12. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learningcited by 6
26 more
  1. 2025Do Retrieval Augmented Language Models Know When They Don't Know?cited by 5
  2. 2025Understanding the Dilemma of Unlearning for Large Language Modelscited by 4
  3. 2025Exploring Consciousness in LLMs: A Systematic Survey of Theories, Implementations, and Frontier Riskscited by 3
  4. 2025Stop Before You Fail: Operational Capability Boundaries for Mitigating Unproductive Reasoning in Large Reasoning Modelscited by 3
  5. 2026Architecting Trust in Artificial Epistemic Agentscited by 3
  6. 2026KBF: Knowledge Boundary as Fingerprint for Language Model and Black-Box API Auditingcited by 3
  7. 2026STRUCTSURVEY: Structured Agentic Retrieval for Automated Survey Paper Generationcited by 3
  8. 2025Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detectioncited by 2
  9. 2026Know More, Know Clearer: A Meta-Cognitive Framework for Knowledge Augmentation in Large Language Modelscited by 2
  10. 2026Neurosymbolic Learning for Inference-Time Argumentationcited by 2
  11. 2025LORE: A Large Generative Model for Search Relevancecited by 1
  12. 2026GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamicscited by 1
  13. 2026What Do Agents Communicate? Characterizing Information Exchange in Multi-Agent Systemscited by 1
  14. 2025ReFactX: Scalable Reasoning with Reliable Facts via Constrained Generationcited by 0
  15. 2025Rethinking LLM Parametric Knowledge as Post-retrieval Confidence for Dynamic Retrieval and Rerankingcited by 0
  16. 2025Assessing the Software Security Comprehension of Large Language Modelscited by 0
  17. 2026Toward IIT-Inspired Consciousness in LLMs: A Reward-Based Learning Frameworkcited by 0
  18. 2026Deep Learning-based Method for Expressing Knowledge Boundary of Black-Box LLMcited by 0
  19. 2026X-RAY: Mapping LLM Reasoning Capability via Formalized and Calibrated Probescited by 0
  20. 2026KA2L: A Knowledge-Aware Active Learning Framework for LLMscited by 0
  21. 2026A Case Study on the Impact of Anonymization Along the RAG Pipelinecited by 0
  22. 2026Localizing RL-Induced Tool Use to a Single Crosscoder Featurecited by 0
  23. 2026CPInj: Uncovering Prompt Injection Risks in Textual Collaborative Prompt Optimizationcited by 0
  24. 2026LittleLearner: Language Models Under Pedagogically Controlled Knowledge Exposurecited by 0
  25. 2026The Geometry of Knowledge Accessibility in Large Language Modelscited by 0
  26. 2026Invisible Ink, Visible Lies: How Production Watermarking Causes LLMs to Hallucinatecited by 0