# Calibrating Small Language Models for Claim Check-Worthiness Detection

arXiv: https://arxiv.org/abs/2608.30731 · HTML: https://mlsanity.com/paper/2608.30731/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 26 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 16 of 16)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [ARES: An Automated Evaluation Framework for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2311.09476/) (2023)
- [Fighting the COVID-19 Infodemic: Modeling the Perspective of Journalists, Fact-Checkers, Social Media Platforms, Policy Makers, and the Society](https://mlsanity.com/paper/2005.00033/) (2020)
- [Large Language Models Help Humans Verify Truthfulness -- Except When They Are Convincingly Wrong](https://mlsanity.com/paper/2310.12558/) (2023)
- [AFaCTA: Assisting the Annotation of Factual Claim Detection with Reliable LLM Annotators](https://mlsanity.com/paper/2402.11073/) (2024)
- [ClaimRank: Detecting Check-Worthy Claims in Arabic and English](https://mlsanity.com/paper/1804.07587/) (2018)
- [Claim Verification in the Age of Large Language Models: A Survey](https://mlsanity.com/paper/2408.14317/) (2024)
- [Do Language Models Mirror Human Confidence? Exploring Psychological Insights to Address Overconfidence in LLMs](https://mlsanity.com/paper/2506.00582/) (2025)
- [A Theoretical Study on Bridging Internal Probability and Self-Consistency for LLM Reasoning](https://mlsanity.com/paper/2510.15444/) (2025)
- [Surprising Efficacy of Fine-Tuned Transformers for Fact-Checking over Larger Language Models](https://mlsanity.com/paper/2402.12147/) (2024)
- [Calibrating Verbalized Confidence with Self-Generated Distractors](https://mlsanity.com/paper/2509.25532/) (2025)
- [Claim Check-Worthiness Detection: How Well do LLMs Grasp Annotation Guidelines?](https://mlsanity.com/paper/2404.12174/) (2024)
- [Large Language Models for Multilingual Previously Fact-Checked Claim Detection](https://mlsanity.com/paper/2503.02737/) (2025)
- [Multilingual and Multi-topical Benchmark of Fine-tuned Language models and Large Language Models for Check-Worthy Claim Detection](https://mlsanity.com/paper/2311.06121/) (2023)
- [Multilingual Fact-Checking at Scale: Fine-Tuned Compact Models vs LLMs](https://mlsanity.com/paper/2606.08605/) (2026)

## Abstract (arXiv)

Assessing claim check-worthiness is an essential first step in automated fact-checking pipelines. This work is motivated by a real deployment challenge at an early-stage startup: running large language models (LLMs) over every incoming claim is cost- and latency-prohibitive, yet smaller models sacrifice accuracy. We propose NN-PPI, a pointwise extension of Prediction-Powered Inference (PPI) that calibrates model predictions at inference time as a lightweight post-hoc layer, without re-training the underlying model. NN-PPI achieves weighted F1 gains ranging from 12% to 33.80% depending on the size and performance of the baseline model, bringing SLMs on par with larger LLMs. Beyond few-shot SLMs, NN-PPI further improves a production-deployed fine-tuned model, demonstrating that residual calibration is complementary to supervised fine-tuning. By recovering LLM-level accuracy from models that are an order of magnitude cheaper to serve, it makes accurate check-worthiness detection substantially cheaper to operate at scale. Our code and data can be found at https://anonymous.4open.science/r/arr-claim-worthiness-F237.