# SC4ANM: Identifying Optimal Section Combinations for Automated Novelty Prediction in Academic Papers

arXiv: https://arxiv.org/abs/2505.16330 · HTML: https://mlsanity.com/paper/2505.16330/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 61 references that are in the map.
- First posted on arXiv 22 May 2025.

## Builds on (most cited first, 24 of 24)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://mlsanity.com/paper/2112.07916/) (2021)
- [Can large language models provide useful feedback on research papers? A large-scale empirical analysis](https://mlsanity.com/paper/2310.01783/) (2023)
- [ReviewerGPT? An Exploratory Study on Using Large Language Models for Paper Reviewing](https://mlsanity.com/paper/2306.00622/) (2023)
- [Large Language Models are Not Yet Human-Level Evaluators for Abstractive Summarization](https://mlsanity.com/paper/2305.13091/) (2023)
- [GPT4 is Slightly Helpful for Peer-Review Assistance: A Pilot Study](https://mlsanity.com/paper/2307.05492/) (2023)
- [Assessing LLMs in Malicious Code Deobfuscation of Real-world Malware Campaigns](https://mlsanity.com/paper/2404.19715/) (2024)
- [Can ChatGPT evaluate research quality?](https://mlsanity.com/paper/2402.05519/) (2024)
- [Enhancing Identification of Structure Function of Academic Articles Using Contextual Information](https://mlsanity.com/paper/2111.14110/) (2021)
- [Creativity in Science and the Link to Cited References: Is the Creative Potential of Papers Reflected in their Cited References?](https://mlsanity.com/paper/1806.00224/) (2018)
- [Which structure of academic articles do referees pay more attention to?: perspective of peer review and full-text of academic articles](https://mlsanity.com/paper/2209.01841/) (2022)
- [Are the confidence scores of reviewers consistent with the review content? Evidence from top conference proceedings in AI](https://mlsanity.com/paper/2505.15031/) (2025)
- [Feature engineering vs. deep learning for paper section identification: Toward applications in Chinese medical literature](https://mlsanity.com/paper/2412.11125/) (2024)
- [What do we know about the disruption index in scientometrics? An overview of the literature](https://mlsanity.com/paper/2308.02383/) (2023)
- [A Content-Based Novelty Measure for Scholarly Publications: A Proof of Concept](https://mlsanity.com/paper/2401.03642/) (2024)

## Built on by (most cited first, 8 of 8)

- [Is this Idea Novel? An Automated Benchmark for Judgment of Research Ideas](https://mlsanity.com/paper/2603.10303/) (2026)
- [OpenNovelty: An LLM-powered Agentic System for Verifiable Scholarly Novelty Assessment](https://mlsanity.com/paper/2601.01576/) (2026)
- [NovBench: Evaluating Large Language Models on Academic Paper Novelty Assessment](https://mlsanity.com/paper/2604.11543/) (2026)
- [OpenLens AI: Fully Autonomous Research Agent for Health Infomatics](https://mlsanity.com/paper/2509.14778/) (2025)
- [Is It Novel and Why? Fine-Grained Patent Novelty Prediction Based on Passage Retrieval](https://mlsanity.com/paper/2605.02392/) (2026)
- [Do Methods Support the Claims? Intra-Paper Verification for Peer Review](https://mlsanity.com/paper/2607.26066/) (2026)
- [Examining the Cognitive Gap Between Authors and Peer Reviewers on Academic Paper Novelty](https://mlsanity.com/paper/2606.13452/) (2026)
- [Which Sections of a Research Paper Best Reveal Its Research Methods? Evidence from Library and Information Science](https://mlsanity.com/paper/2606.19051/) (2026)

## Abstract (arXiv)

Novelty is a core component of academic papers, and there are multiple perspectives on the assessment of novelty. Existing methods often focus on word or entity combinations, which provide limited insights. The content related to a paper's novelty is typically distributed across different core sections, e.g., Introduction, Methodology and Results. Therefore, exploring the optimal combination of sections for evaluating the novelty of a paper is important for advancing automated novelty assessment. In this paper, we utilize different combinations of sections from academic papers as inputs to drive language models to predict novelty scores. We then analyze the results to determine the optimal section combinations for novelty score prediction. We first employ natural language processing techniques to identify the sectional structure of academic papers, categorizing them into introduction, methods, results, and discussion (IMRaD). Subsequently, we used different combinations of these sections (e.g., introduction and methods) as inputs for pretrained language models (PLMs) and large language models (LLMs), employing novelty scores provided by human expert reviewers as ground truth labels to obtain prediction results. The results indicate that using introduction, results and discussion is most appropriate for assessing the novelty of a paper, while the use of the entire text does not yield significant results. Furthermore, based on the results of the PLMs and LLMs, the introduction and results appear to be the most important section for the task of novelty score prediction. The code and dataset for this paper can be accessed at https://github.com/njust-winchy/SC4ANM.