# CiteWorth: Cite-Worthiness Detection for Improved Scientific Document Understanding

arXiv: https://arxiv.org/abs/2105.10912 · HTML: https://mlsanity.com/paper/2105.10912/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 33 references that are in the map.
- First posted on arXiv 23 May 2021.

## Builds on (most cited first, 19 of 19)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Fact or Fiction: Verifying Scientific Claims](https://mlsanity.com/paper/2004.14974/) (2020)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [ScispaCy: Fast and Robust Models for Biomedical Natural Language Processing](https://mlsanity.com/paper/1902.07669/) (2019)
- [Unsupervised Domain Clusters in Pretrained Language Models](https://mlsanity.com/paper/2004.02105/) (2020)
- [A Diagnostic Study of Explainability Techniques for Text Classification](https://mlsanity.com/paper/2009.13295/) (2020)
- [Structural Scaffolds for Citation Intent Classification in Scientific Publications](https://mlsanity.com/paper/1904.01608/) (2019)
- [Construction of the Literature Graph in Semantic Scholar](https://mlsanity.com/paper/1805.02262/) (2018)
- [SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications](https://mlsanity.com/paper/1704.02853/) (2017)
- [A Corpus with Multi-Level Annotations of Patients, Interventions and Outcomes to Support Language Processing for Medical Literature](https://mlsanity.com/paper/1806.04185/) (2018)
- [Investigating the influence of noise and distractors on the interpretation of neural networks](https://mlsanity.com/paper/1611.07270/) (2016)
- [A Supervised Approach to Extractive Summarisation of Scientific Papers](https://mlsanity.com/paper/1706.03946/) (2017)
- [Transformer Based Multi-Source Domain Adaptation](https://mlsanity.com/paper/2009.07806/) (2020)
- [Multi-Task Learning of Keyphrase Boundary Classification](https://mlsanity.com/paper/1704.00514/) (2017)
- [Longitudinal Citation Prediction using Temporal Graph Neural Networks](https://mlsanity.com/paper/2012.05742/) (2020)

## Built on by (most cited first, 17 of 17)

- [Neighborhood Contrastive Learning for Scientific Document Representations with Citation Embeddings](https://mlsanity.com/paper/2202.06671/) (2022)
- [Generating Scientific Claims for Zero-Shot Scientific Fact Checking](https://mlsanity.com/paper/2203.12990/) (2022)
- [Pre-training Multi-task Contrastive Learning Models for Scientific Literature Understanding](https://mlsanity.com/paper/2305.14232/) (2023)
- [Modeling Information Change in Science Communication with Semantically Matched Paraphrases](https://mlsanity.com/paper/2210.13001/) (2022)
- [SsciBERT: A Pre-trained Language Model for Social Science Texts](https://mlsanity.com/paper/2206.04510/) (2022)
- [Generating Media Background Checks for Automated Source Critical Reasoning](https://mlsanity.com/paper/2409.00781/) (2024)
- [Semi-Supervised Exaggeration Detection of Health Science Press Releases](https://mlsanity.com/paper/2108.13493/) (2021)
- [Understanding Fine-grained Distortions in Reports of Scientific Findings](https://mlsanity.com/paper/2402.12431/) (2024)
- [The Next Phase of Scientific Fact-Checking: Advanced Evidence Retrieval from Complex Structured Academic Papers](https://mlsanity.com/paper/2506.20844/) (2025)
- [Determining the Credibility of Science Communication](https://mlsanity.com/paper/2105.14473/) (2021)
- [Did AI get more negative recently?](https://mlsanity.com/paper/2202.13610/) (2022)
- [MIReAD: Simple Method for Learning High-quality Representations from Scientific Documents](https://mlsanity.com/paper/2305.04177/) (2023)
- [Aligning Large Language Model Behavior with Human Citation Preferences](https://mlsanity.com/paper/2602.05205/) (2026)
- [CiteCaseLAW: Citation Worthiness Detection in Caselaw for Legal Assistive Writing](https://mlsanity.com/paper/2305.03508/) (2023)
- [Preface to the Special Issue of the TAL Journal on Scholarly Document Processing](https://mlsanity.com/paper/2506.03587/) (2025)
- [DS@GT at CheckThat! 2025: Exploring Retrieval and Reranking Pipelines for Scientific Claim Source Retrieval on Social Media Discourse](https://mlsanity.com/paper/2507.06563/) (2025)
- [RWGBench: Evaluating Scholarly Positioning in Related Work Generation](https://mlsanity.com/paper/2606.24894/) (2026)

## Abstract (arXiv)

Scientific document understanding is challenging as the data is highly domain specific and diverse. However, datasets for tasks with scientific text require expensive manual annotation and tend to be small and limited to only one or a few fields. At the same time, scientific documents contain many potential training signals, such as citations, which can be used to build large labelled datasets. Given this, we present an in-depth study of cite-worthiness detection in English, where a sentence is labelled for whether or not it cites an external source. To accomplish this, we introduce CiteWorth, a large, contextualized, rigorously cleaned labelled dataset for cite-worthiness detection built from a massive corpus of extracted plain-text scientific documents. We show that CiteWorth is high-quality, challenging, and suitable for studying problems such as domain adaptation. Our best performing cite-worthiness detection model is a paragraph-level contextualized sentence labelling model based on Longformer, exhibiting a 5 F1 point improvement over SciBERT which considers only individual sentences. Finally, we demonstrate that language model fine-tuning with cite-worthiness as a secondary task leads to improved performance on downstream scientific document understanding tasks.