# Predicting Long-Term Citations from Short-Term Linguistic Influence

arXiv: https://arxiv.org/abs/2210.13628 · HTML: https://mlsanity.com/paper/2210.13628/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 56 references that are in the map.
- First posted on arXiv 24 Oct 2022.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [We Need to Talk About Random Splits](https://mlsanity.com/paper/2005.00636/) (2020)
- [Temporal Analysis of Language through Neural Language Models](https://mlsanity.com/paper/1405.3515/) (2014)
- [Time-Out: Temporal Referencing for Robust Modeling of Lexical Semantic Change](https://mlsanity.com/paper/1906.01688/) (2019)
- [UiO-UvA at SemEval-2020 Task 1: Contextualised Embeddings for Lexical Semantic Change Detection](https://mlsanity.com/paper/2005.00050/) (2020)
- [The Social Dynamics of Language Change in Online Networks](https://mlsanity.com/paper/1609.02075/) (2016)
- [Follow the Leader: Documents on the Leading Edge of Semantic Change Get More Citations](https://mlsanity.com/paper/1909.04189/) (2019)
- [Abolitionist Networks: Modeling Language Change in Nineteenth-Century Activist Newspapers](https://mlsanity.com/paper/2103.07538/) (2021)

## Built on by (most cited first, 3 of 3)

- [Words as Gatekeepers: Measuring Discipline-specific Terms and Meanings in Scholarly Publications](https://mlsanity.com/paper/2212.09676/) (2022)
- [Words and Action: Modeling Linguistic Leadership in #BlackLivesMatter Communities](https://mlsanity.com/paper/2412.02637/) (2024)
- [Information Dynamics of Language Communication](https://mlsanity.com/paper/2606.30096/) (2026)

## Abstract (arXiv)

A standard measure of the influence of a research paper is the number of times it is cited. However, papers may be cited for many reasons, and citation count offers limited information about the extent to which a paper affected the content of subsequent publications. We therefore propose a novel method to quantify linguistic influence in timestamped document collections. There are two main steps: first, identify lexical and semantic changes using contextual embeddings and word frequencies; second, aggregate information about these changes into per-document influence scores by estimating a high-dimensional Hawkes process with a low-rank parameter matrix. We show that this measure of linguistic influence is predictive of textitfuture citations: the estimate of linguistic influence from the two years after a paper's publication is correlated with and predictive of its citation count in the following three years. This is demonstrated using an online evaluation with incremental temporal training/test splits, in comparison with a strong baseline that includes predictors for initial citation counts, topics, and lexical features.