# Understanding Fine-grained Distortions in Reports of Scientific Findings

arXiv: https://arxiv.org/abs/2402.12431 · HTML: https://mlsanity.com/paper/2402.12431/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 46 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 16 of 16)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [A Survey on Automated Fact-Checking](https://mlsanity.com/paper/2108.11896/) (2021)
- [Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detection](https://mlsanity.com/paper/2111.07997/) (2021)
- [Best-Worst Scaling More Reliable than Rating Scales: A Case Study on Sentiment Intensity Annotation](https://mlsanity.com/paper/1712.01765/) (2017)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [Scientific Fact-Checking: A Survey of Resources and Approaches](https://mlsanity.com/paper/2305.16859/) (2023)
- [Revise and Resubmit: An Intertextual Model of Text-based Collaboration in Peer Review](https://mlsanity.com/paper/2204.10805/) (2022)
- [CoVERT: A Corpus of Fact-checked Biomedical COVID-19 Tweets](https://mlsanity.com/paper/2204.12164/) (2022)
- [Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications](https://mlsanity.com/paper/2109.14776/) (2021)
- [CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding](https://mlsanity.com/paper/2105.10912/) (2021)
- [Modeling Information Change in Science Communication with Semantically Matched Paraphrases](https://mlsanity.com/paper/2210.13001/) (2022)
- [InfoLossQA: Characterizing and Recovering Information Loss in Text Simplification](https://mlsanity.com/paper/2401.16475/) (2024)
- [Semi-Supervised Exaggeration Detection of Health Science Press Releases](https://mlsanity.com/paper/2108.13493/) (2021)

## Built on by (most cited first, 8 of 8)

- [Missci: Reconstructing Fallacies in Misrepresented Science](https://mlsanity.com/paper/2406.03181/) (2024)
- [Automatic Reviewers Fail to Detect Faulty Reasoning in Research Papers: A New Counterfactual Evaluation Framework](https://mlsanity.com/paper/2508.21422/) (2025)
- [Beyond the Crowd: LLM-Augmented Community Notes for Governing Health Misinformation](https://mlsanity.com/paper/2510.11423/) (2025)
- [From Hypothesis to Publication: A Comprehensive Survey of AI-Driven Research Support Systems](https://mlsanity.com/paper/2503.01424/) (2025)
- [Facts Fade Fast: Evaluating Memorization of Outdated Medical Knowledge in Large Language Models](https://mlsanity.com/paper/2509.04304/) (2025)
- [Counterfactual LLM-based Framework for Measuring Rhetorical Style](https://mlsanity.com/paper/2512.19908/) (2025)
- [Better with Experience: Self-Evolving LLM Agents for Evidence-Grounded Health Community Notes](https://mlsanity.com/paper/2606.02215/) (2026)
- [From 'May' to 'Is': Certainty Distortion in Language Model Rewriting](https://mlsanity.com/paper/2606.07951/) (2026)

## Abstract (arXiv)

Distorted science communication harms individuals and society as it can lead to unhealthy behavior change and decrease trust in scientific institutions. Given the rapidly increasing volume of science communication in recent years, a fine-grained understanding of how findings from scientific publications are reported to the general public, and methods to detect distortions from the original work automatically, are crucial. Prior work focused on individual aspects of distortions or worked with unpaired data. In this work, we make three foundational contributions towards addressing this problem: (1) annotating 1,600 instances of scientific findings from academic papers paired with corresponding findings as reported in news articles and tweets wrt. four characteristics: causality, certainty, generality and sensationalism; (2) establishing baselines for automatically detecting these characteristics; and (3) analyzing the prevalence of changes in these characteristics in both human-annotated and large-scale unlabeled data. Our results show that scientific findings frequently undergo subtle distortions when reported. Tweets distort findings more often than science news reports. Detecting fine-grained distortions automatically poses a challenging task. In our experiments, fine-tuned task-specific models consistently outperform few-shot LLM prompting.