Is Peer-Reviewing Worth the Effort?
Not on the map. Why?
Abstract (arXiv)
How effective is peer-reviewing in identifying important papers? We treat this question as a forecasting task. Can we predict which papers will be highly cited in the future based on venue and "early returns" (citations soon after publication)? We show early returns are more predictive than venue. Finally, we end with constructive suggestions to address scaling challenges: (a) too many submissions and (b) too few qualified reviewers.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 14 of its 52 references that are in the map. First posted on arXiv 18 Dec 2024.
Builds on
14 of 52 references.
References · 14
- 2020Open Graph Benchmark: Datasets for Machine Learning on Graphscited by 2,163
- 2016Enriching Word Vectors with Subword Informationcited by 1,583
- 2017DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networkscited by 1,013
- 2020Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databasescited by 75
- 2021Inconsistency in Conference Peer Review: Revisiting the 2014 NeurIPS Experimentcited by 53
- 2020What Can We Do to Improve Peer Review in NLP?cited by 32
- 2013Quantifying Long-Term Scientific Impactcited by 29
- 2023Peer Reviews of Peer Reviews: A Randomized Controlled Trial and Other Experimentscited by 27
- 2018Predicting citation counts based on deep neural network learning techniquescited by 21
- 2015Predicting the long-term citation impact of recent publicationscited by 8
- 2020Predicting the Citations of Scholarly Papercited by 8
- 2019Predicting long-term publication impact through a combination of early citations and journal impact factorcited by 6
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.