# What Makes Medical Claims (Un)Verifiable? Analyzing Entity and Relation Properties for Fact Verification

arXiv: https://arxiv.org/abs/2402.01360 · HTML: https://mlsanity.com/paper/2402.01360/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 35 references that are in the map.
- First posted on arXiv 2 Feb 2024.

## Builds on (most cited first, 15 of 15)

- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [Explainable Automated Fact-Checking for Public Health Claims](https://mlsanity.com/paper/2010.09926/) (2020)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [SciFact-Open: Towards open-domain scientific claim verification](https://mlsanity.com/paper/2210.13777/) (2022)
- [The Limitations of Stylometry for Detecting Machine-Generated Fake News](https://mlsanity.com/paper/1908.09805/) (2019)
- [Scientific Fact-Checking: A Survey of Resources and Approaches](https://mlsanity.com/paper/2305.16859/) (2023)
- [HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking](https://mlsanity.com/paper/2309.08503/) (2023)
- [What is the Essence of a Claim? Cross-Domain Claim Identification](https://mlsanity.com/paper/1704.07203/) (2017)
- [CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets](https://mlsanity.com/paper/2204.12164/) (2022)
- [Empowering the Fact-checkers! Automatic Identification of Claim Spans on Twitter](https://mlsanity.com/paper/2210.04710/) (2022)
- [Claim Detection in Biomedical Twitter Posts](https://mlsanity.com/paper/2104.11639/) (2021)
- [Extracting Fine-Grained Knowledge Graphs of Scientific Claims: Dataset and Transformer-Based Results](https://mlsanity.com/paper/2109.10453/) (2021)
- [Entity-based Claim Representation Improves Fact-Checking of Medical Content in Tweets](https://mlsanity.com/paper/2209.07834/) (2022)
- [Recovering Patient Journeys: A Corpus of Biomedical Entities and Relations on Twitter (BEAR)](https://mlsanity.com/paper/2204.09952/) (2022)
- [An Entity-based Claim Extraction Pipeline for Real-world Biomedical Fact-checking](https://mlsanity.com/paper/2304.05268/) (2023)

## Built on by (most cited first, 10 of 10)

- [A Survey of Table Reasoning with Large Language Models](https://mlsanity.com/paper/2402.08259/) (2024)
- [The CLEF-2025 CheckThat! Lab: Subjectivity, Fact-Checking, Claim Normalization, and Retrieval](https://mlsanity.com/paper/2503.14828/) (2025)
- [MedCite: Can Language Models Generate Verifiable Text for Medicine?](https://mlsanity.com/paper/2506.06605/) (2025)
- [Can Community Notes Replace Professional Fact-Checkers?](https://mlsanity.com/paper/2502.14132/) (2025)
- [From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems](https://mlsanity.com/paper/2503.01424/) (2025)
- [CORRECT: Context- and Reference-Augmented Reasoning and Prompting for Fact-Checking](https://mlsanity.com/paper/2502.09635/) (2025)
- [A Scoping Review of Natural Language Processing in Addressing Medically Inaccurate Information: Errors, Misinformation, and Hallucination](https://mlsanity.com/paper/2505.00008/) (2025)
- [Self-Adaptive Paraphrasing and Preference Learning for Improved Claim Verifiability](https://mlsanity.com/paper/2412.11653/) (2024)
- [MEVER: Multi-Modal and Explainable Claim Verification with Graph-based Evidence Retrieval](https://mlsanity.com/paper/2602.10023/) (2026)
- [ContextClaim: A Context-Driven Paradigm for Verifiable Claim Detection](https://mlsanity.com/paper/2603.30025/) (2026)

## Abstract (arXiv)

Biomedical claim verification fails if no evidence can be discovered. In these cases, the fact-checking verdict remains unknown and the claim is unverifiable. To improve upon this, we have to understand if there are any claim properties that impact its verifiability. In this work we assume that entities and relations define the core variables in a biomedical claim's anatomy and analyze if their properties help us to differentiate verifiable from unverifiable claims. In a study with trained annotation experts we prompt them to find evidence for biomedical claims, and observe how they refine search queries for their evidence search. This leads to the first corpus for scientific fact verification annotated with subject-relation-object triplets, evidence documents, and fact-checking verdicts (the BEAR-Fact corpus). We find (1) that discovering evidence for negated claims (e.g., X-does-not-cause-Y) is particularly challenging. Further, we see that annotators process queries mostly by adding constraints to the search and by normalizing entities to canonical names. (2) We compare our in-house annotations with a small crowdsourcing setting where we employ medical experts and laypeople. We find that domain expertise does not have a substantial effect on the reliability of annotations. Finally, (3), we demonstrate that it is possible to reliably estimate the success of evidence retrieval purely from the claim text~(.82F), whereas identifying unverifiable claims proves more challenging (.27F). The dataset is available at http://www.ims.uni-stuttgart.de/data/bioclaim.