MSA at SemEval-2025 Task 3: High Quality Weak Labeling and LLM Ensemble Verification for Multilingual Hallucination Detection
Not on the map. Why?
Abstract (arXiv)
This paper describes our submission for SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes. The task involves detecting hallucinated spans in text generated by instruction-tuned Large Language Models (LLMs) across multiple languages. Our approach combines task-specific prompt engineering with an LLM ensemble verification mechanism, where a primary model extracts hallucination spans and three independent LLMs adjudicate their validity through probability-based voting. This framework simulates the human annotation workflow used in the shared task validation and test data. Additionally, fuzzy matching refines span alignment. Our system ranked 1st in Arabic and Basque, 2nd in German, Swedish, and Finnish, and 3rd in Czech, Farsi, and French.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 6 of its 20 references that are in the map. First posted on arXiv 27 May 2025.
Builds on
6 of 20 references.
References · 6
- 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,368
- 2019SpanBERT: Improving Pre-training by Representing and Predicting Spanscited by 834
- 2022Language Models in the Loop: Incorporating Prompting into Weak Supervisioncited by 52
- 2024Comparing Hallucination Detection Metrics for Multilingual Generationcited by 18
- 2024Using Large Language Models for Generating Smart Contracts for Health Insurance from Textual Policiescited by 5
- 2025Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systemscited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.