# Are We Truly Innovating? A Qualitative and Quantitative Study of Originality in AI Research Papers

arXiv: https://arxiv.org/abs/2602.06054 · HTML: https://mlsanity.com/paper/2602.06054/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 24 references that are in the map.
- First posted on arXiv 14 Jan 2026.

## Builds on (most cited first, 11 of 11)

- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researchers](https://mlsanity.com/paper/2409.04109/) (2024)
- [LLM Discussion: Enhancing the Creativity of Large Language Models via Discussion Framework and Role-Play](https://mlsanity.com/paper/2405.06373/) (2024)
- [OpenReviewer: A Specialized Large Language Model for Generating Critical Scientific Paper Reviews](https://mlsanity.com/paper/2412.11948/) (2024)
- [NLPeer: A Unified Resource for the Computational Study of Peer Review](https://mlsanity.com/paper/2211.06651/) (2022)
- [Assessing and Understanding Creativity in Large Language Models](https://mlsanity.com/paper/2401.12491/) (2024)
- [Evaluating and Enhancing Large Language Models for Novelty Assessment in Scholarly Publications](https://mlsanity.com/paper/2409.16605/) (2024)
- [Automated Creativity Evaluation for Large Language Models: A Reference-Based Approach](https://mlsanity.com/paper/2504.15784/) (2025)
- [Literature-Grounded Novelty Assessment of Scientific Ideas](https://mlsanity.com/paper/2506.22026/) (2025)
- [A review on the novelty measurements of academic papers](https://mlsanity.com/paper/2501.17456/) (2025)
- [Unveiling the Merits and Defects of LLMs in Automatic Review Generation for Scientific Papers](https://mlsanity.com/paper/2509.19326/) (2025)

## Built on by (most cited first, 1 of 1)

- [Think-Probe-Respond: Improving Large Language Models as Judges of Research Idea Novelty](https://mlsanity.com/paper/2608.25660/) (2026)

## Abstract (arXiv)

Assessing originality in AI research is arguably the most consequential yet least reliable step in peer review. Reviewer judgments of originality remain opaque, inconsistent, and dependent on comparisons to prior work that are often incomplete. In this paper, we present a large-scale, data-driven qualitative and quantitative analysis of research originality based on over 100,000 peer-review reports from leading AI venues, spanning a period of rapid growth in the field. Leveraging structured, semantically retrieved prior work and signals embedded in expert reviewer assessments, we systematically characterize how originality is perceived in practice and identify the key dimensions that most strongly influence novelty judgments. Our analysis yields a fine-grained, evidence-based framework that equips both authors and reviewers with actionable insights into how originality is evaluated. In addition, we evaluate the reliability of current large language model (LLM) agents in assessing originality. We find that these models tend to systematically overestimate novelty and struggle to detect conceptual plagiarism, particularly in the presence of paraphrasing. We release our dataset, trained models, and code at: https://anonymous.4open.science/r/Novelty-Reviewer-365C/.