mlsanityData to 7 Oct 2026

OnionEval: An Unified Evaluation of Fact-conflicting Hallucination for Small-Large Language Models

arXiv 2501.12975 ↗First posted 22 Jan 2025cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are highly capable but require significant computational resources for both training and inference. Within the LLM family, smaller models (those with fewer than 10 billion parameters) also perform well across various tasks. However, these smaller models share similar limitations to their larger counterparts, including the tendency to hallucinate. Despite the existence of many benchmarks to evaluate hallucination in LLMs, few have specifically focused on small LLMs (SLLMs). Additionally, SLLMs show widely varying performance across different benchmarks. In this paper, we introduce OnionEval, a multi-layer structured framework with a specific metric called the context-influence score (CI), designed to effectively assess the fact-conflicting hallucination tendencies of small LLMs across different contextual levels. Our experimental results reveal a key feature of SLLMs: they excel in factual analysis but face challenges with context reasoning. Further investigation shows that a simple Chain-of-Thought strategy can significantly reduce these limitations, improving the practical usefulness of SLLMs in real-world applications.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 11 of its 18 references that are in the map. First posted on arXiv 22 Jan 2025.

Builds on

11 of 18 references.

References · 11

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,919
  4. 2023A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questionscited by 1,582
  5. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
  6. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
  7. 2023Chain-of-Verification Reduces Hallucination in Large Language Modelscited by 317
  8. 2023Generating Benchmarks for Factuality Evaluation of Language Modelscited by 103
  9. 2023Evaluating Hallucinations in Chinese Large Language Modelscited by 33
  10. 2024Hallucination Detection and Hallucination Mitigation: An Investigationcited by 20
  11. 2024Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Modelscited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023HaluEval: A Large-Scale Hallucination Evaluation Benchmark for Large Language Models
  2. 2025Theoretical Foundations and Mitigation of Hallucination in Large Language Models
  3. 2024LongHalQA: Long-Context Hallucination Evaluation for MultiModal Large Language Models
  4. 2025MALM: A Multi-Information Adapter for Large Language Models to Mitigate Hallucination
  5. 2025Triggering Hallucinations in LLMs: A Quantitative Study of Prompt-Induced Hallucination in Large Language Models
  6. 2024THaMES: An End-to-End Tool for Hallucination Mitigation and Evaluation in Large Language Models
  7. 2025From "Hallucination" to "Suture": Insights from Language Philosophy to Enhance Large Language Models
  8. 2025REFIND at SemEval-2025 Task 3: Retrieval-Augmented Factuality Hallucination Detection in Large Language Models