# What We Talk About When We Talk About LMs: Implicit Paradigm Shifts and the Ship of Language Models

arXiv: https://arxiv.org/abs/2407.01929 · HTML: https://mlsanity.com/paper/2407.01929/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 38 references that are in the map.
- First posted on arXiv 2 Jul 2024.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Survey on In-context Learning](https://mlsanity.com/paper/2301.00234/) (2022)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://mlsanity.com/paper/2111.01243/) (2021)
- [COMET: Commonsense Transformers for Automatic Knowledge Graph Construction](https://mlsanity.com/paper/1906.05317/) (2019)
- [Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews](https://mlsanity.com/paper/2403.07183/) (2024)
- [Growth rates of modern science: A latent piecewise growth curve approach to model publication numbers from established and new literature databases](https://mlsanity.com/paper/2012.07675/) (2020)
- [A Bibliometric Review of Large Language Models Research from 2017 to 2023](https://mlsanity.com/paper/2304.02020/) (2023)
- [Identification of Tasks, Datasets, Evaluation Metrics, and Numeric Scores for Scientific Leaderboards Construction](https://mlsanity.com/paper/1906.09317/) (2019)
- [What Do NLP Researchers Believe? Results of the NLP Community Metasurvey](https://mlsanity.com/paper/2208.12852/) (2022)
- [The Elephant in the Room: Analyzing the Presence of Big Tech in Natural Language Processing Research](https://mlsanity.com/paper/2305.02797/) (2023)
- [Examining Citations of Natural Language Processing Literature](https://mlsanity.com/paper/2005.00912/) (2020)
- [We are Who We Cite: Bridges of Influence Between Natural Language Processing and Other Academic Fields](https://mlsanity.com/paper/2310.14870/) (2023)
- [A Diachronic Analysis of Paradigm Shifts in NLP Research: When, How, and Why?](https://mlsanity.com/paper/2305.12920/) (2023)
- [Inheritance patterns in citation networks reveal scientific memes](https://mlsanity.com/paper/1404.3757/) (2014)

## Built on by (most cited first, 2 of 2)

- [Transforming Scholarly Landscapes: Influence of Large Language Models on Academic Fields beyond Computer Science](https://mlsanity.com/paper/2409.19508/) (2024)
- [TeleCom-Bench: How Far Are Large Language Models from Industrial Telecommunication Applications?](https://mlsanity.com/paper/2605.18025/) (2026)

## Abstract (arXiv)

The term Language Models (LMs) as a time-specific collection of models of interest is constantly reinvented, with its referents updated much like the textitShip of Theseus replaces its parts but remains the same ship in essence. In this paper, we investigate this textitShip of Language Models problem, wherein scientific evolution takes the form of continuous, implicit retrofits of key existing terms. We seek to initiate a novel perspective of scientific progress, in addition to the more well-studied emergence of new terms. To this end, we construct the data infrastructure based on recent NLP publications. Then, we perform a series of text-based analyses toward a detailed, quantitative understanding of the use of Language Models as a term of art. Our work highlights how systems and theories influence each other in scientific discourse, and we call for attention to the transformation of this Ship that we all are contributing to.