# Decide less, communicate more: On the construct validity of end-to-end fact-checking in medicine

arXiv: https://arxiv.org/abs/2506.20876 · HTML: https://mlsanity.com/paper/2506.20876/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 59 references that are in the map.
- First posted on arXiv 25 Jun 2025.

## Builds on (most cited first, 26 of 26)

- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Explainable Automated Fact-Checking for Public Health Claims](https://mlsanity.com/paper/2010.09926/) (2020)
- [Fact-Checking Complex Claims with Program-Guided Reasoning](https://mlsanity.com/paper/2305.12744/) (2023)
- [WiCE: Real-World Entailment for Claims in Wikipedia](https://mlsanity.com/paper/2303.01432/) (2023)
- [Complex Claim Verification with Evidence Retrieved in the Wild](https://mlsanity.com/paper/2305.11859/) (2023)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://mlsanity.com/paper/2106.03794/) (2021)
- [Clarify When Necessary: Resolving Ambiguity Through Interaction with LMs](https://mlsanity.com/paper/2311.09469/) (2023)
- [Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questions](https://mlsanity.com/paper/2410.13788/) (2024)
- [Generating Literal and Implied Subquestions to Fact-check Complex Claims](https://mlsanity.com/paper/2205.06938/) (2022)
- [CLAMBER: A Benchmark of Identifying and Clarifying Ambiguous Information Needs in Large Language Models](https://mlsanity.com/paper/2405.12063/) (2024)
- [FaithScore: Fine-grained Evaluations of Hallucinations in Large Vision-Language Models](https://mlsanity.com/paper/2311.01477/) (2023)
- [Molecular Facts: Desiderata for Decontextualization in LLM Fact Verification](https://mlsanity.com/paper/2406.20079/) (2024)
- [Analyzing Dataset Annotation Quality Management in the Wild](https://mlsanity.com/paper/2307.08153/) (2023)
- [A Closer Look at Claim Decomposition](https://mlsanity.com/paper/2403.11903/) (2024)
- [AmbiFC: Fact-Checking Ambiguous Claims with Evidence](https://mlsanity.com/paper/2104.00640/) (2021)
- [Decomposition Dilemmas: Does Claim Decomposition Boost or Burden Fact-Checking Performance?](https://mlsanity.com/paper/2411.02400/) (2024)
- [HealthFC: Verifying Health Claims with Evidence-Based Medical Fact-Checking](https://mlsanity.com/paper/2309.08503/) (2023)
- [The Perils & Promises of Fact-checking with Large Language Models](https://mlsanity.com/paper/2310.13549/) (2023)
- [CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets](https://mlsanity.com/paper/2204.12164/) (2022)
- [Show Me the Work: Fact-Checkers' Requirements for Explainable Automated Fact-Checking](https://mlsanity.com/paper/2502.09083/) (2025)
- [OpenFactCheck: A Unified Framework for Factuality Evaluation of LLMs](https://mlsanity.com/paper/2408.11832/) (2024)
- [Human and technological infrastructures of fact-checking](https://mlsanity.com/paper/2205.10894/) (2022)
- [RedHOT: A Corpus of Annotated Medical Questions, Experiences, and Claims on Social Media](https://mlsanity.com/paper/2210.06331/) (2022)
- [Document-level Claim Extraction and Decontextualisation for Fact-Checking](https://mlsanity.com/paper/2406.03239/) (2024)
- [An Interactive Multi-modal Query Answering System with Retrieval-Augmented Large Language Models](https://mlsanity.com/paper/2407.04217/) (2024)

## Built on by (most cited first, 2 of 2)

- [Faithfulness vs. Safety: Evaluating LLM Behavior Under Counterfactual Medical Evidence](https://mlsanity.com/paper/2601.11886/) (2026)
- [Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Intervention](https://mlsanity.com/paper/2608.21721/) (2026)

## Abstract (arXiv)

Technological progress has led to concrete advancements in tasks that were regarded as challenging, such as automatic fact-checking. Interest in adopting these systems for public health and medicine has grown due to the high-stakes nature of medical decisions and challenges in critically appraising a vast and diverse medical literature. Evidence-based medicine connects to every individual, and yet the nature of it is highly technical, rendering the medical literacy of majority users inadequate to sufficiently navigate the domain. Such problems with medical communication ripen the ground for end-to-end fact-checking agents: check a claim against current medical literature and return with an evidence-backed verdict. And yet, such systems remain largely unused. In this position paper, developed with expert input, we present the first study examining how clinical experts verify real claims from social media by synthesizing medical evidence. In searching for this upper-bound, we reveal fundamental challenges in end-to-end fact-checking when applied to medicine: Difficulties connecting claims in the wild to scientific evidence in the form of clinical trials; ambiguities in underspecified claims mixed with mismatched intentions; and inherently subjective veracity labels. We argue that fact-checking should be approached as an interactive communication problem, rather than an end-to-end process.