# Measuring Sentence-Level and Aspect-Level (Un)certainty in Science Communications

arXiv: https://arxiv.org/abs/2109.14776 · HTML: https://mlsanity.com/paper/2109.14776/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 3 of its 74 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 3 of 3)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)

## Built on by (most cited first, 27 of 27)

- [Cognitive Mirage: A Review of Hallucinations in Large Language Models](https://mlsanity.com/paper/2309.06794/) (2023)
- [POTATO: The Portable Text Annotation Tool](https://mlsanity.com/paper/2212.08620/) (2022)
- [Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark](https://mlsanity.com/paper/2305.14938/) (2023)
- [Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models](https://mlsanity.com/paper/2305.13712/) (2023)
- [When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Dataset](https://mlsanity.com/paper/2306.06826/) (2023)
- [Modeling Information Change in Science Communication with Semantically Matched Paraphrases](https://mlsanity.com/paper/2210.13001/) (2022)
- [Missci: Reconstructing Fallacies in Misrepresented Science](https://mlsanity.com/paper/2406.03181/) (2024)
- [CIF-Bench: A Chinese Instruction-Following Benchmark for Evaluating the Generalizability of Large Language Models](https://mlsanity.com/paper/2402.13109/) (2024)
- [Can Large Language Models Detect Misinformation in Scientific News Reporting?](https://mlsanity.com/paper/2402.14268/) (2024)
- [Understanding Fine-grained Distortions in Reports of Scientific Findings](https://mlsanity.com/paper/2402.12431/) (2024)
- [Teaching Probabilistic Logical Reasoning to Transformers](https://mlsanity.com/paper/2305.13179/) (2023)
- [Epistemic Integrity in Large Language Models](https://mlsanity.com/paper/2411.06528/) (2024)
- [Inference to the Best Explanation in Large Language Models](https://mlsanity.com/paper/2402.10767/) (2024)
- [On the Rigour of Scientific Writing: Criteria, Analysis, and Insights](https://mlsanity.com/paper/2410.04981/) (2024)
- [Did AI get more negative recently?](https://mlsanity.com/paper/2202.13610/) (2022)
- [Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AI](https://mlsanity.com/paper/2505.15031/) (2025)
- [RIGOURATE: Quantifying Scientific Exaggeration with Evidence-Aligned Claim Evaluation](https://mlsanity.com/paper/2601.04350/) (2026)
- [Communication styles and reader preferences of LLM- and human-authored COVID-19 information explanations: a case study](https://mlsanity.com/paper/2505.08143/) (2025)
- [Modeling and Detecting Company Risks from News: A Case Study in Bloomberg News](https://mlsanity.com/paper/2508.10927/) (2025)
- [Counterfactual LLM-based Framework for Measuring Rhetorical Style](https://mlsanity.com/paper/2512.19908/) (2025)
- [TAME: Token Attribution and Masking for Emergent misalignment](https://mlsanity.com/paper/2609.16754/) (2026)
- [A Test of Time: Predicting the Sustainable Success of Online Collaboration in Wikipedia](https://mlsanity.com/paper/2410.19150/) (2024)
- [Measuring and Analyzing Subjective Uncertainty in Scientific Communications](https://mlsanity.com/paper/2503.21114/) (2025)
- [Modeling Public Perceptions of Science in Media](https://mlsanity.com/paper/2506.16622/) (2025)
- [LLM Assertiveness can be Mechanistically Decomposed into Emotional and Logical Components](https://mlsanity.com/paper/2508.17182/) (2025)
- [From 'May' to 'Is': Certainty Distortion in Language Model Rewriting](https://mlsanity.com/paper/2606.07951/) (2026)
- [Does Finetuning with Scientific Data Increase Hallucinations? A Multi-domain Factuality Evaluation of LLMs](https://mlsanity.com/paper/2606.21359/) (2026)

## Abstract (arXiv)

Certainty and uncertainty are fundamental to science communication. Hedges have widely been used as proxies for uncertainty. However, certainty is a complex construct, with authors expressing not only the degree but the type and aspects of uncertainty in order to give the reader a certain impression of what is known. Here, we introduce a new study of certainty that models both the level and the aspects of certainty in scientific findings. Using a new dataset of 2167 annotated scientific findings, we demonstrate that hedges alone account for only a partial explanation of certainty. We show that both the overall certainty and individual aspects can be predicted with pre-trained language models, providing a more complete picture of the author's intended communication. Downstream analyses on 431K scientific findings from news and scientific abstracts demonstrate that modeling sentence-level and aspect-level certainty is meaningful for areas like science communication. Both the model and datasets used in this paper are released at https://blablablab.si.umich.edu/projects/certainty/.