# Collaboration or Corporate Capture? Quantifying NLP's Reliance on Industry Artifacts and Contributions

arXiv: https://arxiv.org/abs/2312.03912 · HTML: https://mlsanity.com/paper/2312.03912/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 41 references that are in the map.
- First posted on arXiv 6 Dec 2023.

## Builds on (most cited first, 14 of 14)

- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [AI and the Everything in the Whole Wide World Benchmark](https://mlsanity.com/paper/2111.15366/) (2021)
- [What Will it Take to Fix Benchmarking in Natural Language Understanding?](https://mlsanity.com/paper/2104.02145/) (2021)
- [Show Your Work: Improved Reporting of Experimental Results](https://mlsanity.com/paper/1909.03004/) (2019)
- [With Little Power Comes Great Responsibility](https://mlsanity.com/paper/2010.06595/) (2020)
- [The Cost of Training NLP Models: A Concise Overview](https://mlsanity.com/paper/2004.08900/) (2020)
- [Scientific Credibility of Machine Translation Research: A Meta-Evaluation of 769 Papers](https://mlsanity.com/paper/2106.15195/) (2021)
- [The De-democratization of AI: Deep Learning and the Compute Divide in Artificial Intelligence Research](https://mlsanity.com/paper/2010.15581/) (2020)
- [The Grey Hoodie Project: Big Tobacco, Big Tech, and the threat on academic integrity](https://mlsanity.com/paper/2009.13676/) (2020)
- [deep-significance - Easy and Meaningful Statistical Significance Testing in the Age of Neural Networks](https://mlsanity.com/paper/2204.06815/) (2022)
- [What Do NLP Researchers Believe? Results of the NLP Community Metasurvey](https://mlsanity.com/paper/2208.12852/) (2022)
- [The Elephant in the Room: Analyzing the Presence of Big Tech in Natural Language Processing Research](https://mlsanity.com/paper/2305.02797/) (2023)
- [We are Who We Cite: Bridges of Influence Between Natural Language Processing and Other Academic Fields](https://mlsanity.com/paper/2310.14870/) (2023)
- [Why Comparing Single Performance Scores Does Not Allow to Draw Conclusions About Machine Learning Approaches](https://mlsanity.com/paper/1803.09578/) (2018)

## Built on by (most cited first, 2 of 2)

- [Big AI is accelerating the metacrisis: What can we do?](https://mlsanity.com/paper/2512.24863/) (2025)
- [Beyond Good Intentions: When Does the Framing of Multilingual and Low-Resource NLP Research Become a Caricature?](https://mlsanity.com/paper/2608.30866/) (2026)

## Abstract (arXiv)

Impressive performance of pre-trained models has garnered public attention and made news headlines in recent years. Almost always, these models are produced by or in collaboration with industry. Using them is critical for competing on natural language processing (NLP) benchmarks and correspondingly to stay relevant in NLP research. We surveyed 100 papers published at EMNLP 2022 to determine the degree to which researchers rely on industry models, other artifacts, and contributions to publish in prestigious NLP venues and found that the ratio of their citation is at least three times greater than what would be expected. Our work serves as a scaffold to enable future researchers to more accurately address whether: 1) Collaboration with industry is still collaboration in the absence of an alternative or 2) if NLP inquiry has been captured by the motivations and research direction of private corporations.