mlsanityData to 14 Oct 2026

Teaching Large Language Models to Express Knowledge Boundary from Their Own Signals

arXiv 2406.10881 ↗First posted 16 Jun 2024cs.CLcited by 32

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) have achieved great success, but their occasional content fabrication, or hallucination, limits their practical application. Hallucination arises because LLMs struggle to admit ignorance due to inadequate training on knowledge boundaries. We call it a limitation of LLMs that they can not accurately express their knowledge boundary, answering questions they know while admitting ignorance to questions they do not know. In this paper, we aim to teach LLMs to recognize and express their knowledge boundary, so they can reduce hallucinations caused by fabricating when they do not know. We propose CoKE, which first probes LLMs' knowledge boundary via internal confidence given a set of questions, and then leverages the probing results to elicit the expression of the knowledge boundary. Extensive experiments show CoKE helps LLMs express knowledge boundaries, answering known questions while declining unknown ones, significantly improving in-domain and out-of-domain performance.

Key facts. Cited by 32 papers in the map (data through 14 Oct 2026). Builds on 26 of its 45 references that are in the map. First posted on arXiv 16 Jun 2024.

Builds on

26 of 45 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  5. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  6. 2020Learning to summarize from human feedbackcited by 2,588
  7. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  8. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  9. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,336
  10. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  11. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  12. 2022When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memoriescited by 1,128
14 more
  1. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  2. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  3. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  4. 2021Editing Factual Knowledge in Language Modelscited by 553
  5. 2023Physics of Language Models: Part 3.1, Knowledge Storage and Extractioncited by 270
  6. 2023Do Large Language Models Know What They Don't Know?cited by 243
  7. 2024Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?cited by 232
  8. 2023Fine-tuning Language Models for Factualitycited by 221
  9. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 206
  10. 2023LM vs LM: Detecting Factual Errors via Cross Examinationcited by 156
  11. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 147
  12. 2024Unfamiliar Finetuning Examples Control How Language Models Hallucinatecited by 87
  13. 2024Calibrating Large Language Models Using Their Generations Onlycited by 65
  14. 2023Smooth ECE: Principled Reliability Diagrams via Kernel Smoothingcited by 63

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 16; 2026: 10.

32 papers.

Sort:

By area:

  1. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  2. 2025SMART: Self-Aware Agent for Tool Overuse Mitigationcited by 52
  3. 2025Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Modelscited by 52
  4. 2024Internal Consistency and Self-Feedback in Large Language Models: A Surveycited by 50
  5. 2024Knowledge Boundary of Large Language Models: A Surveycited by 40
  6. 2024A Survey on the Honesty of Large Language Modelscited by 23
  7. 2025Chain-of-Thought Prompting Obscures Hallucination Cues in Large Language Models: An Empirical Evaluationcited by 21
  8. 2025LoVeC: Reinforcement Learning for Better Verbalized Confidence in Long-Form Generationscited by 17
  9. 2026Knowing When to Abstain: Medical LLMs Under Clinical Uncertaintycited by 15
  10. 2025Enhancing LLM Reliability via Explicit Knowledge Boundary Modelingcited by 13
  11. 2025GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigationcited by 11
  12. 2025UNCLE: Benchmarking Uncertainty Expressions in Long-Form Generationcited by 11
20 more
  1. 2025Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoningcited by 8
  2. 2025Can LLMs Estimate Student Struggles? Human-AI Difficulty Alignment with Proficiency Simulation for Item Difficulty Predictioncited by 7
  3. 2026Model-Adaptive Tool Necessity Reveals the Knowing-Doing Gap in LLM Tool Usecited by 7
  4. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learningcited by 6
  5. 2025Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Abstention Rewardscited by 6
  6. 2026Reasoning Models Will Sometimes Lie About Their Reasoningcited by 6
  7. 2025ConfRAG: Confidence-Guided Retrieval-Augmenting Generationcited by 4
  8. 2024Tell Me What You Don't Know: Enhancing Refusal Capabilities of Role-Playing Agents via Representation Space Analysis and Editingcited by 3
  9. 2025MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awarenesscited by 3
  10. 2026BAPO: Boundary-Aware Policy Optimization for Reliable Agentic Searchcited by 3
  11. 2024Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusalcited by 1
  12. 2025Learning Auxiliary Tasks Improves Reference-Free Hallucination Detection in Open-Domain Long-Form Generationcited by 1
  13. 2026Knowledge Boundary Discovery for Large Language Modelscited by 1
  14. 2026Learning from AVA: Early Lessons from a Curated and Trustworthy Generative AI for Policy and Development Researchcited by 1
  15. 2026Diagnosing LLM Arbitration Behavior over Pre-evidence Epistemic States in RAG-based Fact-Checkingcited by 1
  16. 2024KBM: Delineating Knowledge Boundary for Adaptive Retrieval in Large Language Modelscited by 0
  17. 2025Assessing the Software Security Comprehension of Large Language Modelscited by 0
  18. 2026Deep Learning-based Method for Expressing Knowledge Boundary of Black-Box LLMcited by 0
  19. 2026Automatically Evolving Prompt Guidelines for Task-Specific Optimizationcited by 0
  20. 2026Rewarding Efficient Reasoning Improves Abstention on Underspecified Tasks in Reasoning Modelscited by 0