"AGI" team at SHROOM-CAP: Data-Centric Approach to Multilingual Hallucination Detection using XLM-RoBERTa
Not on the map. Why?
Abstract (arXiv)
The detection of hallucinations in multilingual scientific text generated by Large Language Models (LLMs) presents significant challenges for reliable AI systems. This paper describes our submission to the SHROOM-CAP 2025 shared task on scientific hallucination detection across 9 languages. Unlike most approaches that focus primarily on model architecture, we adopted a data-centric strategy that addressed the critical issue of training data scarcity and imbalance. We unify and balance five existing datasets to create a comprehensive training corpus of 124,821 samples (50% correct, 50% hallucinated), representing a 172x increase over the original SHROOM training data. Our approach fine-tuned XLM-RoBERTa-Large with 560 million parameters on this enhanced dataset, achieves competitive performance across all languages, including 2nd place in Gujarati (zero-shot language) with Factuality F1 of 0.5107, and rankings between 4th-6th place across the remaining 8 languages. Our results demonstrate that systematic data curation can significantly outperform architectural innovations alone, particularly for low-resource languages in zero-shot settings.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 18 references that are in the map. First posted on arXiv 23 Nov 2025.
Builds on
9 of 18 references.
References · 9
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
- 2020On Faithfulness and Factuality in Abstractive Summarizationcited by 900
- 2023FreshLLMs: Refreshing Large Language Models with Search Engine Augmentationcited by 318
- 2023Data-centric Artificial Intelligence: A Surveycited by 203
- 2023FactCHD: Benchmarking Fact-Conflicting Hallucination Detectioncited by 29
- 2025TabularARGN: A Flexible and Efficient Auto-Regressive Framework for Generating High-Fidelity Synthetic Datacited by 14
- 2025Confabulations from ACL Publications (CAP): A Dataset for Scientific Hallucination Detectioncited by 2
Built on by
Not cited yet.