mlsanityData to 14 Oct 2026

Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models

arXiv 2305.13712 ↗First posted 23 May 2023cs.CL cs.AIcited by 61

Not on the map. Why?

Abstract (arXiv)

This paper investigates the capabilities of Large Language Models (LLMs) in the context of understanding their knowledge and uncertainty over questions. Specifically, we focus on addressing known-unknown questions, characterized by high uncertainty due to the absence of definitive answers. To facilitate our study, we collect a new dataset with Known-Unknown Questions (KUQ) and establish a categorization framework to clarify the origins of uncertainty in such queries. Subsequently, we examine the performance of open-source LLMs, fine-tuned using this dataset, in distinguishing between known and unknown queries within open-ended question-answering scenarios. The fine-tuned models demonstrated a significant improvement, achieving a considerable increase in F1-score relative to their pre-fine-tuning state. Through a comprehensive analysis, we reveal insights into the models' improved uncertainty articulation and their consequent efficacy in multi-agent debates. These findings help us understand how LLMs can be trained to identify and express uncertainty, improving our knowledge of how they understand and express complex or unclear information.

Key facts. Cited by 61 papers in the map (data through 14 Oct 2026). Builds on 34 of its 61 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

34 of 61 references.

References · 34

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  6. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  9. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  10. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  11. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,754
  12. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
22 more
  1. 2018Deep contextualized word representationscited by 2,290
  2. 2021SimCSE: Simple Contrastive Learning of Sentence Embeddingscited by 2,173
  3. 2022Survey of Hallucination in Natural Language Generationcited by 2,075
  4. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2018CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledgecited by 1,822
  7. 2018Know What You Don't Know: Unanswerable Questions for SQuADcited by 1,554
  8. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  9. 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,067
  10. 2019How Can We Know What Language Models Know?cited by 959
  11. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  12. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  13. 2019ELI5: Long Form Question Answeringcited by 575
  14. 2022Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Modelcited by 559
  15. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  16. 2023Benchmarking Large Language Models for News Summarizationcited by 445
  17. 2020AmbigQA: Answering Ambiguous Open-domain Questionscited by 390
  18. 2021On Hallucination and Predictive Uncertainty in Conditional Language Generationcited by 176
  19. 2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
  20. 2022(QA)2: Question Answering with Questionable Assumptionscited by 51
  21. 2021Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communicationscited by 27
  22. 2023ControversialQA: Exploring Controversy in Question Answeringcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 16; 2025: 18; 2026: 23.

61 papers.

Sort:

By area:

  1. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  2. 2024Large Language Model-Brained GUI Agents: A Surveycited by 162
  3. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  4. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 121
  5. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 105
  6. 2024The Art of Saying No: Contextual Noncompliance in Language Modelscited by 105
  7. 2025HalluLens: LLM Hallucination Benchmarkcited by 92
  8. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
  9. 2023Alignment for Honestycited by 74
  10. 2025Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?cited by 58
  11. 2024Uncertainty-Based Abstention in LLMs Improves Safety and Reduces Hallucinationscited by 54
  12. 2024Characterizing LLM Abstention Behavior in Science QA with Context Perturbationscited by 26
38 more
  1. 2024A Survey on the Honesty of Large Language Modelscited by 23
  2. 2024Perception of Knowledge Boundary for Large Language Models through Semi-open-ended Question Answeringcited by 20
  3. 2025Large language models for artificial general intelligence (AGI): A survey of foundational principles and approachescited by 18
  4. 2024WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledgecited by 16
  5. 2026Agentic Abstention: Do Agents Know When to Stop Instead of Act?cited by 15
  6. 2024Refusal Tokens: A Simple Way to Calibrate Refusals in Large Language Modelscited by 14
  7. 2025Language Model Uncertainty Quantification with Attention Chaincited by 14
  8. 2025Query-Level Uncertainty in Large Language Modelscited by 14
  9. 2025Detecting (Un)answerability in Large Language Models with Linear Directionscited by 13
  10. 2025Do Large Language Models Know How Much They Know?cited by 11
  11. 2025MMBoundary: Advancing MLLM Knowledge Boundary Awareness through Reasoning Step Confidence Calibrationcited by 11
  12. 2026Over-Searching in Search-Augmented Large Language Modelscited by 11
  13. 2025Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safetycited by 10
  14. 2025A Desideratum for Conversational Agents: Capabilities, Challenges, and Future Directionscited by 9
  15. 2025Uncertainty-Aware Large Language Models for Explainable Disease Diagnosiscited by 9
  16. 2025RefusalBench: Generative Evaluation of Selective Refusal in Grounded Language Modelscited by 9
  17. 2024Meaningful Learning: Enhancing Abstract Reasoning in Large Language Models via Generic Fact Guidancecited by 8
  18. 2024When to Speak, When to Abstain: Contrastive Decoding with Abstentioncited by 8
  19. 2025Language Models Can Predict Their Own Behaviorcited by 8
  20. 2026Knowing When to Quit: A Principled Framework for Dynamic Abstention in LLM Reasoningcited by 8
  21. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 7
  22. 2025Addressing Pitfalls in the Evaluation of Uncertainty Estimation Methods for Natural Language Generationcited by 7
  23. 2026To Call or Not to Call: A Framework to Assess and Optimize LLM Tool Callingcited by 7
  24. 2023Large Language Models as Source Planner for Personalized Knowledge-grounded Dialoguecited by 6
  25. 2024Rethinking the Uncertainty: A Critical Review and Analysis in the Era of Large Language Modelscited by 6
  26. 2024Beyond Introspection: Reinforcing Thinking via Externalist Behavioral Feedbackcited by 5
  27. 2023ZhuJiu: A Multi-dimensional, Multi-faceted Chinese Benchmark for Large Language Modelscited by 4
  28. 2026The Yes-Man Syndrome: Benchmarking Abstention in Embodied Robotic Agentscited by 4
  29. 2026SciPredict: Can LLMs Predict the Outcomes of Scientific Experiments in Natural Sciences?cited by 3
  30. 2026When Should LLMs Search? Counterfactual Supervision for Search Routingcited by 3
  31. 2026Neurosymbolic Learning for Inference-Time Argumentationcited by 2
  32. 2026HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoningcited by 2
  33. 2026Do Large Language Models Know What They Don't Know II? A Fully Behavioral, Non-Cognitive Measure of Epistemic Honestycited by 2
  34. 2025Thinking About Thinking: Evaluating Reasoning in Post-Trained Language Modelscited by 1
  35. 2026UCPO: Uncertainty-Aware Policy Optimizationcited by 1
  36. 2026Uncertainty and Fairness Awareness in LLM-Based Recommendation Systemscited by 1
  37. 2026GRADE: Probing Knowledge Gaps in LLMs through Gradient Subspace Dynamicscited by 1
  38. 2026Enhancing LLM Metacognition via Cognitive Pairwise Trainingcited by 1

50 most cited of 61 All 61 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Uncertainty Quantification for LLM Agents: A Taxonomy, an Evaluation Protocol, and an Empirical Study
  2. 2024MAQA: Evaluating Uncertainty Quantification in LLMs Regarding Data Uncertainty
  3. 2026Debate-on-Graph: Reliable and Adaptive Reasoning of Large Language Model on Uncertain Knowledge Graph
  4. 2025Inside-Out: Hidden Factual Knowledge in LLMs
  5. 2026Estimating Uncertainty from Reasoning: A Large-Scale Study of Multi- and Crosslingual MCQA Performance in LLMs
  6. 2025Tools in the Loop: Quantifying Uncertainty of LLM Question Answering Systems That Use Tools
  7. 2024Can Knowledge Graphs Make Large Language Models More Trustworthy? An Empirical Study Over Open-ended Question Answering
  8. 2025Evaluating Large Language Model with Knowledge Oriented Language Specific Simple Question Answering