mlsanityData to 14 Oct 2026

Halo: Estimation and Reduction of Hallucinations in Open-Source Weak Large Language Models

arXiv 2308.11764 ↗First posted 22 Aug 2023cs.CL cs.AIcited by 36

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) have revolutionized Natural Language Processing (NLP). Although convenient for research and practical applications, open-source LLMs with fewer parameters often suffer from severe hallucinations compared to their larger counterparts. This paper focuses on measuring and reducing hallucinations in BLOOM 7B, a representative of such weaker open-source LLMs that are publicly available for research and commercial applications. We introduce HaloCheck, a lightweight BlackBox knowledge-free framework designed to quantify the severity of hallucinations in LLMs. Additionally, we explore techniques like knowledge injection and teacher-student approaches to alleviate hallucinations in low-parameter LLMs. Our experiments effectively demonstrate the reduction of hallucinations in challenging domains for these LLMs.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Builds on 37 of its 53 references that are in the map. First posted on arXiv 22 Aug 2023.

Builds on

37 of 53 references.

References · 37

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  7. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  8. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  9. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  10. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  11. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  12. 2017A Broad-Coverage Challenge Corpus for Sentence Understanding through Inferencecited by 2,682
25 more
  1. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  2. 2020REALM: Retrieval-Augmented Language Model Pre-Trainingcited by 2,141
  3. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  4. 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  6. 2023SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Modelscited by 810
  7. 2020How Much Knowledge Can You Pack Into the Parameters of a Language Model?cited by 719
  8. 2021Retrieval Augmentation Reduces Hallucination in Conversationcited by 692
  9. 2022Automatic Chain of Thought Prompting in Large Language Modelscited by 688
  10. 2023Capabilities of GPT-4 on Medical Challenge Problemscited by 638
  11. 2023How Far Can Camels Go? Exploring the State of Instruction Tuning on Open Resourcescited by 408
  12. 2021SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarizationcited by 407
  13. 2023Augmented Language Models: a Surveycited by 403
  14. 2022Large Language Models Are Reasoning Teacherscited by 329
  15. 2023How Language Model Hallucinations Can Snowballcited by 311
  16. 2022Teaching Small Language Models to Reasoncited by 284
  17. 2021ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generationcited by 260
  18. 2023Specializing Smaller Language Models towards Multi-Step Reasoningcited by 251
  19. 2023Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Frameworkcited by 139
  20. 2023Self-contradictory Hallucinations of Large Language Models: Evaluation, Detection and Mitigationcited by 124
  21. 2022Evaluating the Factual Consistency of Large Language Models Through News Summarizationcited by 92
  22. 2023Boosting Theory-of-Mind Performance in Large Language Models via Promptingcited by 64
  23. 2023MQAG: Multiple-choice Question Answering and Generation for Assessing Information Consistency in Summarizationcited by 30
  24. 2022SKILL: Structured Knowledge Infusion for Large Language Modelscited by 27
  25. 2022Knowledge Prompts: Injecting World Knowledge into Language Models through Soft Promptscited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 18; 2025: 9; 2026: 4.

36 papers.

Sort:

By area:

  1. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 698
  2. 2023A Survey of Hallucination in Large Foundation Modelscited by 368
  3. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 259
  4. 2024When Can LLMs Actually Correct Their Own Mistakes? A Critical Survey of Self-Correction of LLMscited by 240
  5. 2023Can LLM-Generated Misinformation Be Detected?cited by 177
  6. 2025A Comprehensive Survey in LLM(-Agent) Full Stack Safety: Data, Training and Deploymentcited by 116
  7. 2024Factuality of Large Language Models: A Surveycited by 66
  8. 2024Building Guardrails for Large Language Modelscited by 65
  9. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  10. 2024How Easy is It to Fool Your Multimodal LLMs? An Empirical Analysis on Deceptive Promptscited by 47
  11. 2024Multicalibration for Confidence Scoring in LLMscited by 46
  12. 2023Deficiency of Large Language Models in Finance: An Empirical Examination of Hallucinationcited by 41
24 more
  1. 2024Knowledge Overshadowing Causes Amalgamated Hallucination in Large Language Modelscited by 25
  2. 2023UHGEval: Benchmarking the Hallucination of Chinese Large Language Models via Unconstrained Generationcited by 22
  3. 2024Estimating the Hallucination Rate of Generative AIcited by 20
  4. 2024Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Frameworkcited by 19
  5. 2024Can We Verify Step by Step for Incorrect Answer Detection?cited by 15
  6. 2024Collu-Bench: A Benchmark for Predicting Language Model Hallucinations in Codecited by 13
  7. 2025Enhancing Mathematical Reasoning in Large Language Models with Self-Consistency-Based Hallucination Detectioncited by 13
  8. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 8
  9. 2024Integrative Decoding: Improve Factuality via Implicit Self-consistencycited by 8
  10. 2024Evaluating Consistency and Reasoning Capabilities of Large Language Modelscited by 7
  11. 2025Loki's Dance of Illusions: A Comprehensive Survey of Hallucination in Large Language Modelscited by 7
  12. 2025Rote Learning Considered Useful: Generalizing over Memorized Data in LLMscited by 5
  13. 2026MARCH: Multi-Agent Reinforced Self-Check for LLM Hallucinationcited by 5
  14. 2024DAHL: Domain-specific Automated Hallucination Evaluation of Long-Form Text through a Benchmark Dataset in Biomedicinecited by 4
  15. 2024LLMs Can Check Their Own Results to Mitigate Hallucinations in Traffic Understanding Taskscited by 3
  16. 2024Can Generative AI Solve Your In-Context Learning Problem? A Martingale Perspectivecited by 2
  17. 2025Detecting LLM Fact-conflicting Hallucinations Enhanced by Temporal-logic-based Reasoningcited by 2
  18. 2024LaMsS: When Large Language Models Meet Self-Skepticismcited by 1
  19. 2025Atomic Consistency Preference Optimization for Long-Form Question Answeringcited by 1
  20. 2025Hallucination Detection and Mitigation with Diffusion in Multi-Variate Time-Series Foundation Modelscited by 0
  21. 2025Sample, Align, Synthesize: Graph-Based Response Synthesis with ConGrscited by 0
  22. 2026SLM Finetuning for Natural Language to Domain Specific Code Generation in Productioncited by 0
  23. 2026Grounding Multi-Hop Reasoning in Structural Causal Models via Group Relative Policy Optimizationcited by 0
  24. 2026Hallucinations in LLMs: A Lifecycle-Based Survey of Causes, Detection, Mitigation, and Preventioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
  2. 2024HaluEval-Wild: Evaluating Hallucinations of Language Models in the Wild
  3. 2025Theoretical Foundations and Mitigation of Hallucination in Large Language Models
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
  5. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMs
  6. 2026Evaluating Hallucinations in Domain-Adapted Large Language Models
  7. 2024THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation in Large Language Models
  8. 2026Scalable Token-Level Hallucination Detection in Large Language Models