DSC2025 -- ViHallu Challenge: Detecting Hallucination in Vietnamese LLMs
Not on the map. Why?
Abstract (arXiv)
The reliability of large language models (LLMs) in production environments remains significantly constrained by their propensity to generate hallucinations -- fluent, plausible-sounding outputs that contradict or fabricate information. While hallucination detection has recently emerged as a priority in English-centric benchmarks, low-to-medium resource languages such as Vietnamese remain inadequately covered by standardized evaluation frameworks. This paper introduces the DSC2025 ViHallu Challenge, the first large-scale shared task for detecting hallucinations in Vietnamese LLMs. We present the ViHallu dataset, comprising 10,000 annotated triplets of (context, prompt, response) samples systematically partitioned into three hallucination categories: no hallucination, intrinsic, and extrinsic hallucinations. The dataset incorporates three prompt types -- factual, noisy, and adversarial -- to stress-test model robustness. A total of 111 teams participated, with the best-performing system achieving a macro-F1 score of 84.80%, compared to a baseline encoder-only score of 32.83%, demonstrating that instruction-tuned LLMs with structured prompting and ensemble strategies substantially outperform generic architectures. However, the gap to perfect performance indicates that hallucination detection remains a challenging problem, particularly for intrinsic (contradiction-based) hallucinations. This work establishes a rigorous benchmark and explores a diverse range of detection methodologies, providing a foundation for future research into the trustworthiness and reliability of Vietnamese language AI systems.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 5 of its 11 references that are in the map. First posted on arXiv 8 Jan 2026.
Builds on
5 of 11 references.
References · 5
- 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
- 2018FEVER: a large-scale dataset for Fact Extraction and VERificationcited by 1,375
- 2020PhoBERT: Pre-trained language models for Vietnamesecited by 149
- 2025SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared Task on Hallucinations and Related Observable Overgeneration Mistakescited by 27
- 2024SemEval-2024 Shared Task 6: SHROOM, a Shared-task on Hallucinations and Related Observable Overgeneration Mistakescited by 24
Built on by
Citing papers per year: 2026: 1.
1 paper.