# Assessing Effectiveness of Using Internal Signals for Check-Worthy Claim Identification in Unlabeled Data for Automated Fact-Checking

arXiv: https://arxiv.org/abs/2111.01706 · HTML: https://mlsanity.com/paper/2111.01706/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 20 references that are in the map.
- First posted on arXiv 2 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [FEVER: a large-scale dataset for Fact Extraction and VERification](https://mlsanity.com/paper/1803.05355/) (2018)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://mlsanity.com/paper/1909.03242/) (2019)
- [Generating Fact Checking Explanations](https://mlsanity.com/paper/2004.05773/) (2020)
- [Fine-grained Fact Verification with Kernel Graph Attention Network](https://mlsanity.com/paper/1910.09796/) (2019)
- [GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification](https://mlsanity.com/paper/1908.01843/) (2019)
- [Automated Fact Checking: Task formulations, methods and future directions](https://mlsanity.com/paper/1806.07687/) (2018)
- [Coreferential Reasoning Learning for Language Representation](https://mlsanity.com/paper/2004.06870/) (2020)
- [Towards Automated Factchecking: Developing an Annotation Schema and Benchmark for Consistent Automated Claim Detection](https://mlsanity.com/paper/1809.08193/) (2018)
- [ClaimRank: Detecting Check-Worthy Claims in Arabic and English](https://mlsanity.com/paper/1804.07587/) (2018)
- [Extractive and Abstractive Explanations for Fact-Checking and Evaluation of News](https://mlsanity.com/paper/2104.12918/) (2021)
- [Self-Supervised Claim Identification for Automated Fact Checking](https://mlsanity.com/paper/2102.02335/) (2021)

## Built on by (most cited first, 1 of 1)

- [Claim Detection for Automated Fact-checking: A Survey on Monolingual, Multilingual and Cross-Lingual Research](https://mlsanity.com/paper/2401.11969/) (2024)

## Abstract (arXiv)

While recent work on automated fact-checking has focused mainly on verifying and explaining claims, for which the list of claims is readily available, identifying check-worthy claim sentences from a text remains challenging. Current claim identification models rely on manual annotations for each sentence in the text, which is an expensive task and challenging to conduct on a frequent basis across multiple domains. This paper explores methodology to identify check-worthy claim sentences from fake news articles, irrespective of domain, without explicit sentence-level annotations. We leverage two internal supervisory signals - headline and the abstractive summary - to rank the sentences based on semantic similarity. We hypothesize that this ranking directly correlates to the check-worthiness of the sentences. To assess the effectiveness of this hypothesis, we build pipelines that leverage the ranking of sentences based on either the headline or the abstractive summary. The top-ranked sentences are used for the downstream fact-checking tasks of evidence retrieval and the article's veracity prediction by the pipeline. Our findings suggest that the top 3 ranked sentences contain enough information for evidence-based fact-checking of a fake news article. We also show that while the headline has more gisting similarity with how a fact-checking website writes a claim, the summary-based pipeline is the most promising for an end-to-end fact-checking system.