# On the Use of Context for Predicting Citation Worthiness of Sentences in Scholarly Articles

arXiv: https://arxiv.org/abs/2104.08962 · HTML: https://mlsanity.com/paper/2104.08962/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 18 Apr 2021.

## Builds on (most cited first, 8 of 8)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [Passage Re-ranking with BERT](https://mlsanity.com/paper/1901.04085/) (2019)
- [Complex Question Answering: Unsupervised Learning Approaches and Experiments](https://mlsanity.com/paper/1401.3479/) (2014)

## Built on by (most cited first, 2 of 2)

- [KG-CTG: Citation Generation through Knowledge Graph-guided Large Language Models](https://mlsanity.com/paper/2404.09763/) (2024)
- [CiteCaseLAW: Citation Worthiness Detection in Caselaw for Legal Assistive Writing](https://mlsanity.com/paper/2305.03508/) (2023)

## Abstract (arXiv)

In this paper, we study the importance of context in predicting the citation worthiness of sentences in scholarly articles. We formulate this problem as a sequence labeling task solved using a hierarchical BiLSTM model. We contribute a new benchmark dataset containing over two million sentences and their corresponding labels. We preserve the sentence order in this dataset and perform document-level train/test splits, which importantly allows incorporating contextual information in the modeling process. We evaluate the proposed approach on three benchmark datasets. Our results quantify the benefits of using context and contextual embeddings for citation worthiness. Lastly, through error analysis, we provide insights into cases where context plays an essential role in predicting citation worthiness.