# SChuBERT: Scholarly Document Chunks with BERT-encoding boost Citation Count Prediction

arXiv: https://arxiv.org/abs/2012.11740 · HTML: https://mlsanity.com/paper/2012.11740/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Dec 2020.

## Built on by (most cited first, 9 of 9)

- [Automated scholarly paper review: Concepts, technologies, and challenges](https://mlsanity.com/paper/2111.07533/) (2021)
- [SsciBERT: A Pre-trained Language Model for Social Science Texts](https://mlsanity.com/paper/2206.04510/) (2022)
- [Automatic Evaluation Metrics for Artificially Generated Scientific Research](https://mlsanity.com/paper/2503.05712/) (2025)
- [CiteBART: Learning to Generate Citations for Local Citation Recommendation](https://mlsanity.com/paper/2412.17534/) (2024)
- [H2CGL: Modeling Dynamics of Citation Network for Impact Prediction](https://mlsanity.com/paper/2305.01572/) (2023)
- [MultiSChuBERT: Effective Multimodal Fusion for Scholarly Document Quality Prediction](https://mlsanity.com/paper/2308.07971/) (2023)
- [CiMaTe: Citation Count Prediction Effectively Leveraging the Main Text](https://mlsanity.com/paper/2410.04404/) (2024)
- [ForeCite: Adapting Pre-Trained Language Models to Predict Future Citation Rates of Academic Papers](https://mlsanity.com/paper/2505.08941/) (2025)
- [What do writing features tell us about AI papers?](https://mlsanity.com/paper/2107.06310/) (2021)

## Abstract (arXiv)

Predicting the number of citations of scholarly documents is an upcoming task in scholarly document processing. Besides the intrinsic merit of this information, it also has a wider use as an imperfect proxy for quality which has the advantage of being cheaply available for large volumes of scholarly documents. Previous work has dealt with number of citations prediction with relatively small training data sets, or larger datasets but with short, incomplete input text. In this work we leverage the open access ACL Anthology collection in combination with the Semantic Scholar bibliometric database to create a large corpus of scholarly documents with associated citation information and we propose a new citation prediction model called SChuBERT. In our experiments we compare SChuBERT with several state-of-the-art citation prediction models and show that it outperforms previous methods by a large margin. We also show the merit of using more training data and longer input for number of citations prediction.