# SciImpact: A Multi-Dimensional, Multi-Field Benchmark for Scientific Impact Prediction

arXiv: https://arxiv.org/abs/2604.17141 · HTML: https://mlsanity.com/paper/2604.17141/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 49 references that are in the map.
- First posted on arXiv 18 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [SciBERT: A Pretrained Language Model for Scientific Text](https://mlsanity.com/paper/1903.10676/) (2019)
- [OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and concepts](https://mlsanity.com/paper/2205.01833/) (2022)
- [Construction of the Literature Graph in Semantic Scholar](https://mlsanity.com/paper/1805.02262/) (2018)
- [Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoning](https://mlsanity.com/paper/2512.20848/) (2025)
- [Reduced, Reused and Recycled: The Life of a Dataset in Machine Learning Research](https://mlsanity.com/paper/2112.01716/) (2021)
- [A Comprehensive Survey of Scientific Large Language Models and Their Applications in Scientific Discovery](https://mlsanity.com/paper/2406.10833/) (2024)
- [A Web-scale system for scientific knowledge exploration](https://mlsanity.com/paper/1805.12216/) (2018)
- [Quantifying Long-Term Scientific Impact](https://mlsanity.com/paper/1306.3293/) (2013)
- [Forecasting high-impact research topics via machine learning on evolving knowledge graphs](https://mlsanity.com/paper/2402.08640/) (2024)
- [The Effect of Metadata on Scientific Literature Tagging: A Cross-Field Cross-Model Study](https://mlsanity.com/paper/2302.03341/) (2023)
- [From Words to Worth: Newborn Article Impact Prediction with LLM](https://mlsanity.com/paper/2408.03934/) (2024)
- [A Century of Science: Globalization of Scientific Collaborations, Citations, and Innovations](https://mlsanity.com/paper/1704.05150/) (2017)
- [From Newborn to Impact: Bias-Aware Citation Prediction](https://mlsanity.com/paper/2510.19246/) (2025)
- [Overton -- A bibliometric database of policy document citations](https://mlsanity.com/paper/2201.07643/) (2022)
- [Will This Paper Increase Your h-index? Scientific Impact Prediction](https://mlsanity.com/paper/1412.4754/) (2014)
- [Scientific Prizes and the Extraordinary Growth of Scientific Topics](https://mlsanity.com/paper/2012.09269/) (2020)
- [Quantifying perceived impact of scientific publications](https://mlsanity.com/paper/1612.03962/) (2016)
- [Modeling and Predicting Citation Count via Recurrent Neural Network with Long Short-Term Memory](https://mlsanity.com/paper/1811.02129/) (2018)
- [FOS: A Large-Scale Temporal Graph Benchmark for Scientific Interdisciplinary Link Prediction](https://mlsanity.com/paper/2511.18631/) (2025)

## Built on by (most cited first, 2 of 2)

- [ForeSci: Evaluating LLM Agents for Forward-Looking AI Research Judgment](https://mlsanity.com/paper/2606.00644/) (2026)
- [Learning to Ideate for Scientific Impact](https://mlsanity.com/paper/2609.29802/) (2026)

## Abstract (arXiv)

The rapid growth of scientific literature calls for automated methods to assess and predict research impact. Prior work has largely focused on citation-based metrics, leaving limited evaluation of models' capability to reason about other impact dimensions. To this end, we introduce SciImpact, a large-scale, multi-dimensional benchmark for scientific impact prediction spanning 19 fields. SciImpact captures various forms of scientific influence, ranging from citation counts to award recognition, media attention, patent reference, and artifact adoption, by integrating heterogeneous data sources and targeted web crawling. It comprises 215,928 contrastive paper pairs reflecting meaningful impact differences in both short-term (e.g., Best Paper Award) and long-term settings (e.g., Nobel Prize). We evaluate 11 widely used large language models (LLMs) on SciImpact. Results show that off-the-shelf models exhibit substantial variability across dimensions and fields, while multi-task supervised fine-tuning consistently enables smaller LLMs (e.g., 4B) to markedly outperform much larger models (e.g., 30B) and surpass powerful closed-source LLMs (e.g., o4-mini). These results establish SciImpact as a challenging benchmark and demonstrate its value for multi-dimensional, multi-field scientific impact prediction. Our project homepage is https://flypig23.github.io/sciimpact-homepage/