# Is Peer-Reviewing Worth the Effort?

arXiv: https://arxiv.org/abs/2412.14351 · HTML: https://mlsanity.com/paper/2412.14351/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 52 references that are in the map.
- First posted on arXiv 18 Dec 2024.

## Builds on (most cited first, 14 of 14)

- [Open Graph Benchmark: Datasets for Machine Learning on Graphs](https://mlsanity.com/paper/2005.00687/) (2020)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks](https://mlsanity.com/paper/1704.04110/) (2017)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experiment](https://mlsanity.com/paper/2109.09774/) (2021)
- [What Can We Do to Improve Peer Review in NLP?](https://mlsanity.com/paper/2010.03863/) (2020)
- [Quantifying Long-Term Scientific Impact](https://mlsanity.com/paper/1306.3293/) (2013)
- [Peer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Experiments](https://mlsanity.com/paper/2311.09497/) (2023)
- [Predicting citation counts based on deep neural network learning techniques](https://mlsanity.com/paper/1809.04365/) (2018)
- [Predicting the long-term citation impact of recent publications](https://mlsanity.com/paper/1503.09156/) (2015)
- [Predicting the Citations of Scholarly Paper](https://mlsanity.com/paper/2008.05013/) (2020)
- [Predicting long-term publication impact through a combination of early citations and journal impact factor](https://mlsanity.com/paper/1909.08907/) (2019)
- [Prediction of highly cited papers](https://mlsanity.com/paper/1310.8220/) (2013)
- [How to improve the prediction based on citation impact percentiles for years shortly after the publication date?](https://mlsanity.com/paper/1311.4731/) (2013)

## Built on by (most cited first, 2 of 2)

- [Position: The Artificial Intelligence and Machine Learning Community Should Adopt a More Transparent and Regulated Peer Review Process](https://mlsanity.com/paper/2502.00874/) (2025)
- [How Much Were You Told? Measuring External Information in Peer Reviews](https://mlsanity.com/paper/2609.28041/) (2026)

## Abstract (arXiv)

How effective is peer-reviewing in identifying important papers? We treat this question as a forecasting task. Can we predict which papers will be highly cited in the future based on venue and "early returns" (citations soon after publication)? We show early returns are more predictive than venue. Finally, we end with constructive suggestions to address scaling challenges: (a) too many submissions and (b) too few qualified reviewers.