mlsanityData to 7 Oct 2026

Do Large Language Models Know What They Don't Know?

arXiv 2305.18153 ↗First posted 29 May 2023cs.CLcited by 238

Not on the map. Why?

Cited by 31 in LLM reasoning, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Large language models (LLMs) have a wealth of knowledge that allows them to excel in various Natural Language Processing (NLP) tasks. Current research focuses on enhancing their performance within their existing knowledge. Despite their vast knowledge, LLMs are still limited by the amount of information they can accommodate and comprehend. Therefore, the ability to understand their own limitations on the unknows, referred to as self-knowledge, is of paramount importance. This study aims to evaluate LLMs' self-knowledge by assessing their ability to identify unanswerable or unknowable questions. We introduce an automated methodology to detect uncertainty in the responses of these models, providing a novel measure of their self-knowledge. We further introduce a unique dataset, SelfAware, consisting of unanswerable questions from five diverse categories and their answerable counterparts. Our extensive analysis, involving 20 LLMs including GPT-3, InstructGPT, and LLaMA, discovering an intrinsic capacity for self-knowledge within these models. Moreover, we demonstrate that in-context learning and instruction tuning can further enhance this self-knowledge. Despite this promising insight, our findings also highlight a considerable gap between the capabilities of these models and human proficiency in recognizing the limits of their knowledge.

Key facts. Cited by 238 papers in the map (data through 7 Oct 2026). Builds on 20 of its 30 references that are in the map. First posted on arXiv 29 May 2023.

Builds on

20 of 30 references.

References · 20

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,155
  10. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  11. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,168
  12. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
8 more
  1. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  2. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,538
  3. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,397
  4. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,359
  5. 2023PaLM 2 Technical Reportcited by 1,154
  6. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,057
  7. 2022Complexity-Based Prompting for Multi-Step Reasoningcited by 387
  8. 2016NewsQA: A Machine Comprehension Datasetcited by 330

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 26; 2024: 75; 2025: 70; 2026: 67.

238 papers.

Sort:

By area:

  1. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
  2. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  3. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 253
  4. 2023PromptAgent: Strategic Planning with Language Models Enables Expert-level Prompt Optimizationcited by 226
  5. 2025Why Language Models Hallucinatecited by 205
  6. 2023Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificitycited by 204
  7. 2023Navigate through Enigmatic Labyrinth A Survey of Chain of Thought Reasoning: Advances, Frontiers and Futurecited by 184
  8. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  9. 2024Large Legal Fictions: Profiling Legal Hallucinations in Large Language Modelscited by 144
  10. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 129
  11. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  12. 2023Towards Accurate Differential Diagnosis with Large Language Modelscited by 115
38 more
  1. 2024What Large Language Models Know and What People Think They Knowcited by 109
  2. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 99
  3. 2025HalluLens: LLM Hallucination Benchmarkcited by 88
  4. 2023Factcheck-Bench: Fine-Grained Evaluation Benchmark for Automatic Fact-checkerscited by 86
  5. 2024FaithEval: Can Your Language Model Stay Faithful to Context, Even If "The Moon is Made of Marshmallows"cited by 85
  6. 2023Towards Mitigating Hallucination in Large Language Models via Self-Reflectioncited by 81
  7. 2024LUQ: Long-text Uncertainty Quantification for LLMscited by 79
  8. 2023Boosting Language Models Reasoning with Chain-of-Knowledge Promptingcited by 77
  9. 2023Alignment for Honestycited by 72
  10. 2023Corex: Pushing the Boundaries of Complex Reasoning through Multi-Model Collaborationcited by 70
  11. 2023Self-Knowledge Guided Retrieval Augmentation for Large Language Modelscited by 69
  12. 2024Sufficient Context: A New Lens on Retrieval Augmented Generation Systemscited by 69
  13. 2024How Large Language Models Encode Context Knowledge? A Layer-Wise Probing Studycited by 64
  14. 2024When Do LLMs Need Retrieval Augmentation? Mitigating LLMs' Overconfidence Helps Retrieval Augmentationcited by 59
  15. 2024Factual Confidence of LLMs: on Reliability and Robustness of Current Estimatorscited by 59
  16. 2025The MASK Benchmark: Disentangling Honesty From Accuracy in AI Systemscited by 57
  17. 2025Thought Crime: Backdoors and Emergent Misalignment in Reasoning Modelscited by 57
  18. 2024Aligning Language Models to Explicitly Handle Ambiguitycited by 55
  19. 2024Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinationscited by 51
  20. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 49
  21. 2024BLoB: Bayesian Low-Rank Adaptation by Backpropagation for Large Language Modelscited by 48
  22. 2024Can AI Assistants Know What They Don't Know?cited by 47
  23. 2024A Comprehensive Evaluation of Quantization Strategies for Large Language Modelscited by 47
  24. 2024Calibrating the Confidence of Large Language Models by Eliciting Fidelitycited by 47
  25. 2023Testing Language Model Agents Safely in the Wildcited by 45
  26. 2024On Large Language Models' Hallucination with Regard to Known Factscited by 43
  27. 2024Benchmarking Hallucination in Large Language Models based on Unanswerable Math Word Problemcited by 42
  28. 2024Detecting Hallucinations in Large Language Model Generation: A Token Probability Approachcited by 40
  29. 2024Knowledge Boundary of Large Language Models: A Surveycited by 36
  30. 2023Evaluating Hallucinations in Chinese Large Language Modelscited by 33
  31. 2023How Do Large Language Models Capture the Ever-changing World Knowledge? A Review of Recent Advancescited by 33
  32. 2023Learn to Refuse: Making Large Language Models More Controllable and Reliable through Knowledge Scope Limitation and Refusal Mechanismcited by 33
  33. 2025Theoretical Physics Benchmark (TPBench) -- a Dataset and Study of AI Reasoning Capabilities in Theoretical Physicscited by 33
  34. 2024Teaching Large Language Models to Express Knowledge Boundary from Their Own Signalscited by 32
  35. 2024LLM-jp: A Cross-organizational Project for the Research and Development of Fully Open Japanese LLMscited by 31
  36. 2024Self-DC: When to Reason and When to Act? Self Divide-and-Conquer for Compositional Unknown Questionscited by 30
  37. 2023FactCHD: Benchmarking Fact-Conflicting Hallucination Detectioncited by 29
  38. 2023Trends in Integration of Knowledge and Large Language Models: A Survey and Taxonomy of Methods, Benchmarks, and Applicationscited by 28

50 most cited of 238 All 238 (JSON)