mlsanityData to 7 Oct 2026

Learning to Trust Your Feelings: Leveraging Self-awareness in LLMs for Hallucination Mitigation

arXiv 2401.15449 ↗First posted 27 Jan 2024cs.CLcited by 43

Not on the map. Why?

Abstract (arXiv)

We evaluate the ability of Large Language Models (LLMs) to discern and express their internal knowledge state, a key factor in countering factual hallucination and ensuring reliable application of LLMs. We observe a robust self-awareness of internal knowledge state in LLMs, evidenced by over 85% accuracy in knowledge probing. However, LLMs often fail to express their internal knowledge during generation, leading to factual hallucinations. We develop an automated hallucination annotation tool, Dreamcatcher, which merges knowledge probing and consistency checking methods to rank factual preference data. Using knowledge preference as reward, We propose a Reinforcement Learning from Knowledge Feedback (RLKF) training framework, leveraging reinforcement learning to enhance the factuality and honesty of LLMs. Our experiments across multiple models show that RLKF training effectively enhances the ability of models to utilize their internal knowledge state, boosting performance in a variety of knowledge-based and honesty-related tasks.

Key facts. Cited by 43 papers in the map (data through 7 Oct 2026). Builds on 26 of its 33 references that are in the map. First posted on arXiv 27 Jan 2024.

Builds on

26 of 33 references.

References · 26

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Training language models to follow instructions with human feedbackcited by 15,712
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  4. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  5. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  6. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  7. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  8. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  9. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  10. 2023Qwen Technical Reportcited by 3,273
  11. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
14 more
  1. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,259
  2. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  3. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
  4. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  5. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 712
  6. 2023C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Modelscited by 694
  7. 2023The Internal State of an LLM Knows When It's Lyingcited by 644
  8. 2023CMMLU: Measuring massive multitask language understanding in Chinesecited by 436
  9. 2022Self-critiquing models for assisting human evaluatorscited by 372
  10. 2023How Language Model Hallucinations Can Snowballcited by 310
  11. 2022Factuality Enhanced Language Models for Open-Ended Text Generationcited by 200
  12. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  13. 2023Investigating the Factual Knowledge Boundary of Large Language Models with Retrieval Augmentationcited by 146
  14. 2023Do Language Models Know When They're Hallucinating References?cited by 115

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 20; 2026: 6.

43 papers.

Sort:

By area:

  1. 2024Know Your Limits: A Survey of Abstention in Large Language Modelscited by 116
  2. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 102
  3. 2024Building Guardrails for Large Language Modelscited by 61
  4. 2025HaluMem: Evaluating Hallucinations in Memory Systems of Agentscited by 51
  5. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 45
  6. 2025Entropy-based Exploration Conduction for Multi-step Reasoningcited by 28
  7. 2024BeHonest: Benchmarking Honesty in Large Language Modelscited by 20
  8. 2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learningcited by 20
  9. 2025KnowRL: Exploring Knowledgeable Reinforcement Learning for Factualitycited by 17
  10. 2024DebUnc: Improving Large Language Model Agent Communication With Uncertainty Metricscited by 13
  11. 2024UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Modelscited by 13
  12. 2024Varying Shades of Wrong: Aligning LLMs with Wrong Answers Onlycited by 9
31 more
  1. 2024SEER: Self-Aligned Evidence Extraction for Retrieval-Augmented Generationcited by 9
  2. 2025C-LoRA: Contextual Low-Rank Adaptation for Uncertainty Estimation in Large Language Modelscited by 9
  3. 2024Self-training Large Language Models through Knowledge Detectioncited by 8
  4. 2024On-Policy Self-Alignment with Fine-grained Knowledge Feedback for Hallucination Mitigationcited by 8
  5. 2025BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMscited by 8
  6. 2025Self-Route: Automatic Mode Switching via Capability Estimation for Efficient Reasoningcited by 8
  7. 2024Evaluating Consistency and Reasoning Capabilities of Large Language Modelscited by 7
  8. 2024Leveraging Graph Structures to Detect Hallucinations in Large Language Modelscited by 7
  9. 2025Toward Embodied AGI: A Review of Embodied AI and the Road Aheadcited by 7
  10. 2025AI Awarenesscited by 6
  11. 2025What Makes a Good Natural Language Prompt?cited by 6
  12. 2024Into the Unknown: Self-Learning Large Language Modelscited by 5
  13. 2025KnowRL: Teaching Language Models to Know What They Knowcited by 5
  14. 2025Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinationscited by 5
  15. 2024Evaluating and Enhancing Trustworthiness of LLMs in Perception Taskscited by 4
  16. 2024CLUE: Concept-Level Uncertainty Estimation for Large Language Modelscited by 4
  17. 2025DeLTa: A Decoding Strategy based on Logit Trajectory Prediction Improves Factuality and Reasoning Abilitycited by 3
  18. 2025MAC-Tuning: LLM Multi-Compositional Problem Reasoning with Enhanced Knowledge Boundary Awarenesscited by 3
  19. 2026Scaling with Confidence: Calibrating Confidence of LLMs for Adaptive Test Time Scalingcited by 3
  20. 2025The Illusionist's Prompt: Exposing the Factual Vulnerabilities of Large Language Models with Linguistic Nuancescited by 2
  21. 2025Shaking to Reveal: Perturbation-Based Detection of LLM Hallucinationscited by 2
  22. 2025Context-Aware Pragmatic Metacognitive Prompting for Sarcasm Detectioncited by 2
  23. 2026Structuring the Space of Sociotechnical Alignmentcited by 2
  24. 2024Drawing the Line: Enhancing Trustworthiness of MLLMs Through the Power of Refusalcited by 1
  25. 2024FacLens: Transferable Probe for Foreseeing Non-Factuality in Fact-Seeking Question Answering of Large Language Modelscited by 0
  26. 2025CPR: Mitigating Large Language Model Hallucinations with Curative Prompt Refinementcited by 0
  27. 2025Mathematical Analysis of Hallucination Dynamics in Large Language Models: Uncertainty Quantification, Advanced Decoding, and Principled Mitigationcited by 0
  28. 2026Not All Needles Are Found: How Fact Distribution and Don't Make It Up Prompts Shape Retrieval, Reasoning, and Hallucination in Long-Context LLMscited by 0
  29. 2026Attribution Techniques for Mitigating Hallucinated Information in RAG Systems: A Surveycited by 0
  30. 2026Listen to the Layers: Mitigating Hallucinations with Inter-Layer Disagreementcited by 0
  31. 2026FAITH: Factuality Alignment through Integrating Trustworthiness and Honestnesscited by 0