# MSA at SemEval-2025 Task 3: High Quality Weak Labeling and LLM Ensemble Verification for Multilingual Hallucination Detection

arXiv: https://arxiv.org/abs/2505.20880 · HTML: https://mlsanity.com/paper/2505.20880/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 6 of its 20 references that are in the map.
- First posted on arXiv 27 May 2025.

## Builds on (most cited first, 6 of 6)

- [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019)
- [SpanBERT: Improving Pre-training by Representing and Predicting Spans](https://mlsanity.com/paper/1907.10529/) (2019)
- [Language Models in the Loop: Incorporating Prompting into Weak Supervision](https://mlsanity.com/paper/2205.02318/) (2022)
- [Comparing Hallucination Detection Metrics for Multilingual Generation](https://mlsanity.com/paper/2402.10496/) (2024)
- [Using Large Language Models for Generating Smart Contracts for Health Insurance from Textual Policies](https://mlsanity.com/paper/2407.07019/) (2024)
- [Few-Shot Optimized Framework for Hallucination Detection in Resource-Limited NLP Systems](https://mlsanity.com/paper/2501.16616/) (2025)

## Built on by (most cited first, 1 of 1)

- [Improving Answer Extraction in Context-based Question Answering Systems Using LLMs](https://mlsanity.com/paper/2606.06197/) (2026)

## Abstract (arXiv)

This paper describes our submission for SemEval-2025 Task 3: Mu-SHROOM, the Multilingual Shared-task on Hallucinations and Related Observable Overgeneration Mistakes. The task involves detecting hallucinated spans in text generated by instruction-tuned Large Language Models (LLMs) across multiple languages. Our approach combines task-specific prompt engineering with an LLM ensemble verification mechanism, where a primary model extracts hallucination spans and three independent LLMs adjudicate their validity through probability-based voting. This framework simulates the human annotation workflow used in the shared task validation and test data. Additionally, fuzzy matching refines span alignment. Our system ranked 1st in Arabic and Basque, 2nd in German, Swedish, and Finnish, and 3rd in Czech, Farsi, and French.