# A Content-Based Novelty Measure for Scholarly Publications: A Proof of Concept

arXiv: https://arxiv.org/abs/2401.03642 · HTML: https://mlsanity.com/paper/2401.03642/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 41 references that are in the map.
- First posted on arXiv 8 Jan 2024.

## Builds on (most cited first, 4 of 4)

- [A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications](https://mlsanity.com/paper/1804.09635/) (2018)
- [Improving Wikipedia Verifiability with AI](https://mlsanity.com/paper/2207.06220/) (2022)
- [Do we measure novelty when we analyze unusual combinations of cited references? A validation study of bibliometric novelty indicators based on F1000Prime data](https://mlsanity.com/paper/1910.03233/) (2019)
- [Creativity in Science and the Link to Cited References: Is the Creative Potential of Papers Reflected in their Cited References?](https://mlsanity.com/paper/1806.00224/) (2018)

## Built on by (most cited first, 1 of 1)

- [SC4ANM: Identifying Optimal Section Combinations for Automated Novelty Prediction in Academic Papers](https://mlsanity.com/paper/2505.16330/) (2025)

## Abstract (arXiv)

Novelty, akin to gene mutation in evolution, opens possibilities for scholarly advancement. Although peer review remains the gold standard for evaluating novelty in scholarly communication and resource allocation, the vast volume of submissions necessitates an automated measure of scholarly novelty. Adopting a perspective that views novelty as the atypical combination of existing knowledge, we introduce an information-theoretic measure of novelty in scholarly publications. This measure quantifies the degree of 'surprise' perceived by a language model that represents the word distribution of scholarly discourse. The proposed measure is accompanied by face and construct validity evidence; the former demonstrates correspondence to scientific common sense, and the latter is endorsed through alignment with novelty evaluations from a select panel of domain experts. Additionally, characterized by its interpretability, fine granularity, and accessibility, this measure addresses gaps prevalent in existing methods. We believe this measure holds great potential to benefit editors, stakeholders, and policymakers, and it provides a reliable lens for examining the relationship between novelty and academic dynamics such as creativity, interdisciplinarity, and scientific advances.