NCL-UoR at SemEval-2025 Task 3: Detecting Multilingual Hallucination and Related Observable Overgeneration Text Spans with Modified RefChecker and Modified SeflCheckGPT
Not on the map. Why?
Abstract (arXiv)
SemEval-2025 Task 3 (Mu-SHROOM) focuses on detecting hallucinations in content generated by various large language models (LLMs) across multiple languages. This task involves not only identifying the presence of hallucinations but also pinpointing their specific occurrences. To tackle this challenge, this study introduces two methods: modified RefChecker and modified SelfCheckGPT. The modified RefChecker integrates prompt-based factual verification into References, structuring them as claim-based tests rather than single external knowledge sources. The modified SelfCheckGPT incorporates external knowledge to overcome its reliance on internal knowledge. In addition, both methods' original prompt designs are enhanced to identify hallucinated words within LLM-generated texts. Experimental results demonstrate the effectiveness of the approach, achieving a high ranking on the test dataset in detecting hallucinations across various languages, with an average IoU of 0.5310 and an average COR of 0.5669.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 6 of its 18 references that are in the map. First posted on arXiv 2 Mar 2025.
Builds on
6 of 18 references.
References · 6
- 2022Language Models (Mostly) Know What They Knowcited by 1,783
- 2023Hallucinations in Large Multilingual Translation Modelscited by 126
- 2019Assessing The Factual Accuracy of Generated Textcited by 103
- 2021Increasing Faithfulness in Knowledge-Grounded Dialogue with Controllable Featurescited by 74
- 2023Chainpoll: A high efficacy method for LLM hallucination detectioncited by 39
- 2021WikiCheck: An end-to-end open source Automatic Fact-Checking API based on Wikipediacited by 12
Built on by
Citing papers per year: 2025: 1.
1 paper.