More Computational Resources Do Not Ensure Higher Scholarly Impact: Evidence from Leading NLP Conference Papers
Not on the map. Why?
Abstract (arXiv)
Computational resources are increasingly central to NLP research, but how closely reported GPU capability aligns with scholarly impact remains unclear. We analyze 13,921 ACL, EMNLP, and NAACL main-conference papers published between 2020 and 2025, using GPU resources as our operational measure of computational resources. From full texts, we extract GPU models and counts, standardize each paper's largest reported configuration into a comparable hardware-capability measure, and link these data to citation, award, topic, and institutional metadata. GPU reporting became more common but remained incomplete, while reported capability increased mainly through newer hardware generations and medium-scale multi-GPU configurations. Resource concentration substantially exceeded impact concentration: the annual top 20% of GPU-quantifiable papers accounted for 83.9%-89.9% of reported GPU capability, but only 27%-32% of citations and 20%-33% of paper awards. In adjusted models, a tenfold increase in aggregate reported GPU capability was associated with a 3.52-percentage-point increase in within-NLP topic-year citation percentile, but increased model R^2 by only 0.0042. GPU count showed more consistent positive associations with citation and award outcomes than newer hardware generation. Overall, reported GPU resources are associated with scholarly impact but provide little standalone explanation of research influence.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 38 references that are in the map. First posted on arXiv 22 Aug 2026.
Builds on
25 of 38 references.
References · 25
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
- 2020Scaling Laws for Neural Language Modelscited by 6,586
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2020GShard: Scaling Giant Models with Conditional Computation and Automatic Shardingcited by 1,631
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,254
- 2022BERTopic: Neural topic modeling with a class-based TF-IDF procedurecited by 801
- 2019Green AIcited by 627
- 2021Carbon Emissions and Large Neural Network Trainingcited by 567
- 2019Quantifying the Carbon Emissions of Machine Learningcited by 521
13 more
- 2020Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stoppingcited by 384
- 2024MinerU: An Open-Source Solution for Precise Document Content Extractioncited by 267
- 2022Estimating the Carbon Footprint of BLOOM, a 176B Parameter Language Modelcited by 265
- 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 249
- 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 191
- 2022OpenAlex: A fully-open index of scholarly works, authors, venues, institutions, and conceptscited by 177
- 2019Show Your Work: Improved Reporting of Experimental Resultscited by 169
- 2022Compute Trends Across Three Eras of Machine Learningcited by 163
- 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
- 2020The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Researchcited by 67
- 2024The Compute Divide in Machine Learning: A Threat to Academic Contribution and Scrutiny?cited by 20
- 2023Reproducibility in NLP: What Have We Learned from the Checklist?cited by 18
- 2023NLP Reproducibility For All: Understanding Experiences of Beginnerscited by 4
Built on by
Not cited yet.