mlsanityData to 7 Oct 2026

HalluVerse25: Fine-grained Multilingual Benchmark Dataset for LLM Hallucinations

arXiv 2503.07833 ↗First posted 10 Mar 2025cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are increasingly used in various contexts, yet remain prone to generating non-factual content, commonly referred to as "hallucinations". The literature categorizes hallucinations into several types, including entity-level, relation-level, and sentence-level hallucinations. However, existing hallucination datasets often fail to capture fine-grained hallucinations in multilingual settings. In this work, we introduce HalluVerse25, a multilingual LLM hallucination dataset that categorizes fine-grained hallucinations in English, Arabic, and Turkish. Our dataset construction pipeline uses an LLM to inject hallucinations into factual biographical sentences, followed by a rigorous human annotation process to ensure data quality. We evaluate several LLMs on HalluVerse25, providing valuable insights into how proprietary models perform in detecting LLM-generated hallucinations across different contexts.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 10 Mar 2025.

Builds on

16 of 22 references.

References · 16

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2024The Llama 3 Herd of Modelscited by 14,192
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  5. 2023Mistral 7Bcited by 3,065
  6. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  7. 2024Phi-4 Technical Reportcited by 675
  8. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  9. 2023A Stitch in Time Saves Nine: Detecting and Mitigating Hallucinations of LLMs by Validating Low-Confidence Generationcited by 202
  10. 2023Sources of Hallucination by Large Language Models on Inference Taskscited by 168
  11. 2023Multilingual Machine Translation with Large Language Models: Empirical Results and Analysiscited by 156
  12. 2024Fine-grained Hallucination Detection and Editing for Language Modelscited by 146
4 more
  1. 2023On Learning to Summarize with Large Language Models as Referencescited by 80
  2. 2024LLMs Will Always Hallucinate, and We Need to Live With Thiscited by 77
  3. 2023Diving Deep into Modes of Fact Hallucinations in Dialogue Systemscited by 32
  4. 2024Analyzing LLM Behavior in Dialogue Summarization: Unveiling Circumstantial Hallucination Trendscited by 25

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 9.

13 papers.

Sort:

By area:

  1. 2025AraHalluEval: A Fine-grained Hallucination Evaluation Framework for Arabic LLMscited by 12
  2. 2025Evaluating Arabic Large Language Models: A Survey of Benchmarks, Methods, and Gapscited by 9
  3. 2026Do LLM hallucination detectors suffer from low-resource effect?cited by 3
  4. 2026PRISM: Probing Reasoning, Instruction, and Source Memory in LLM Hallucinationscited by 3
  5. 2025AraTable: Benchmarking LLMs' Reasoning and Understanding of Arabic Tabular Datacited by 2
  6. 2026HalluScore: Large Language Model Hallucination Question Answering Benchmarkcited by 2
  7. 2026HalluTruthQA: A Fine-Grained Benchmark for Hallucination Detection, Localization, and Explanation in Arabic Question Answeringcited by 2
  8. 2026ALPS: A Diagnostic Challenge Set for Arabic Linguistic & Pragmatic Reasoningcited by 1
  9. 2026BenHalluEval: A Multi-Task Hallucination Evaluation Framework for Large Language Models on Bengalicited by 1
  10. 2026Does Bielik Know What It Doesn't Know? Activation Dispersion Separates Entity Familiarity from Factual Reliability Across Model Scalecited by 1
  11. 2026Lost in Speech: Trilingual Spoken Hallucination Detection Across Audio and Transcriptscited by 1
  12. 2025FIBER: A Multilingual Evaluation Resource for Factual Inference Biascited by 0
1 more
  1. 2026Physics as the label for measuring and correcting materials reasoning in multimodal modelscited by 0