# Modelling Language using Large Language Models

arXiv: https://arxiv.org/abs/2404.09579 · HTML: https://mlsanity.com/paper/2404.09579/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 47 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 10 of 10)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems](https://mlsanity.com/paper/1905.00537/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Language models are not naysayers: An analysis of language models on negation benchmarks](https://mlsanity.com/paper/2306.08189/) (2023)
- [On the proper role of linguistically-oriented deep net analysis in linguistic theorizing](https://mlsanity.com/paper/2106.08694/) (2021)
- [Dodrio: Exploring Transformer Models with Interactive Visualization](https://mlsanity.com/paper/2103.14625/) (2021)
- [Contextualized Word Embeddings Encode Aspects of Human-Like Word Sense Knowledge](https://mlsanity.com/paper/2010.13057/) (2020)

## Built on by (most cited first, 2 of 2)

- [Large Language Models and Cognitive Science: A Comprehensive Review of Similarities, Differences, and Challenges](https://mlsanity.com/paper/2409.02387/) (2024)
- [Cross-Cultural Fashion Design via Interactive Large Language Models and Diffusion Models](https://mlsanity.com/paper/2501.15571/) (2025)

## Abstract (arXiv)

This paper argues that large language models have a valuable scientific role to play in serving as scientific models of public languages. Linguistic study should not only be concerned with the cognitive processes behind linguistic competence, but also with language understood as an external, social entity. Once this is recognized, the value of large language models as scientific models becomes clear. This paper defends the position against a number of arguments to the effect that language models provide no linguistic insight. Building upon Weisberg's (2007) notion of a model construal, it is then argued that recent work in computational linguistics to better understand the inner workings of large language models can be used to develop a model construal for large language models as models of a language.