DS@GT ARC at CheckThat! 2026: LLM-Based Trace Ranking and Grouped Reward Modeling for Multilingual Numerical Claim Verification
On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning
Abstract (arXiv)
Automated verification of numerical claims is a challenging problem, as it requires both language understanding and quantitative reasoning. This paper describes our system for CLEF 2026 CheckThat! Task 2, which focuses on ranking reasoning traces generated by large language models (LLMs) and predicting a final verdict for numerical claims in English and Arabic. We explore two approaches. The first approach fine-tunes an LLM-based verifier using LoRA to score each reasoning trace independently as a binary classification problem, and selects the final verdict using Best-of-N selection. We further experiment with adaptive sub-claim decomposition to break complex claims into simpler parts before verification. The second approach uses a lightweight TF-IDF reward model with handcrafted numeric and temporal overlap features to score traces, and aggregates scores by verdict group to determine the final prediction. For Arabic, we compare a general multilingual model against AraBERT, a language-specific model pretrained on Arabic text. Our results show that the LLM-based approach outperforms the lightweight reward model on most metrics, particularly Recall@5, while the reward-based approach shows stronger performance on the Conflicting class. Sub-claim decomposition did not improve performance, suggesting that claim splitting introduces noise rather than aiding reasoning. For Arabic, AraBERT outperforms the multilingual baseline across most metrics.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 11 of its 18 references that are in the map. First posted on arXiv 27 Jul 2026.
Builds on
11 of 18 references.
References · 11
- 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,259
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
- 2020AraBERT: Transformer-based Model for Arabic Language Understandingcited by 359
- 2023WiCE: Real-World Entailment for Claims in Wikipediacited by 118
- 2020Overview of CheckThat! 2020: Automatic Identification and Verification of Claims in Social Mediacited by 46
- 2018Overview of the CLEF-2018 CheckThat! Lab on Automatic Identification and Verification of Political Claims. Task 1: Check-Worthinesscited by 41
- 2024A Closer Look at Claim Decompositioncited by 40
- 2024QuanTemp: A real-world open-domain benchmark for fact-checking numerical claimscited by 22
- 2021Assisting the Human Fact-Checkers: Detecting All Previously Fact-Checked Claims in a Documentcited by 19
- 2026GR2: Generative Reasoning Re-rankercited by 11
- 2025DS@GT at CheckThat! 2025: Evaluating Context and Tokenization Strategies for Numerical Fact Verificationcited by 2
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Claim Verification in the Age of Large Language Models: A Survey
- 2025Veri-R1: Toward Precise and Faithful Claim Verification via Online Reinforcement Learning
- 2025CompassVerifier: A Unified and Robust Verifier for LLMs Evaluation and Outcome Reward
- 2026Towards Comprehensive Stage-wise Benchmarking of Large Language Models in Fact-Checking
- 2024IAI Group at CheckThat! 2024: Transformer Models and Data Augmentation for Checkworthy Claim Detection
- 2025Reasoning-CV: Fine-tuning Powerful Reasoning LLMs for Knowledge-Assisted Claim Verification
- 2025Think Right, Not More: Test-Time Scaling for Numerical Claim Verification
- 2025Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs