# Computational conceptual history of scientific concepts: From early digital methods to LLMs

arXiv: https://arxiv.org/abs/2606.04118 · HTML: https://mlsanity.com/paper/2606.04118/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 76 references that are in the map.
- First posted on arXiv 2 Jun 2026.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [A Survey on Multimodal Large Language Models](https://mlsanity.com/paper/2306.13549/) (2023)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Diachronic word embeddings and semantic shifts: a survey](https://mlsanity.com/paper/1806.03537/) (2018)
- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift](https://mlsanity.com/paper/1912.01072/) (2019)
- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [Contextualized language models for semantic change detection: lessons learned](https://mlsanity.com/paper/2209.00154/) (2022)
- [Can Language Models Represent the Past without Anachronism?](https://mlsanity.com/paper/2505.00030/) (2025)
- [That's sick dude!: Automatic identification of word sense change across different timescales](https://mlsanity.com/paper/1405.4392/) (2014)
- [Co-word Maps and Topic Modeling: A Comparison Using Small and Medium-Sized Corpora (n < 1000)](https://mlsanity.com/paper/1511.03020/) (2015)
- [Meaning at the Planck scale? Contextualized word embeddings for doing history, philosophy, and sociology of science](https://mlsanity.com/paper/2411.14073/) (2024)
- [A comparison of citation-based clustering and topic modeling for science mapping](https://mlsanity.com/paper/2309.06160/) (2023)
- [Astro-HEP-BERT: A bidirectional language model for studying the meanings of concepts in astrophysics and high energy physics](https://mlsanity.com/paper/2411.14877/) (2024)

## Abstract (arXiv)

This article situates large language models (LLMs) within the longer history of computational approaches to concept analysis in the history, philosophy, and sociology of science (HPSS). We examine what LLMs add to existing methods, how they inherit longstanding problems, and review recent case studies that employ them. In the first part, we reconstruct computational conceptual history before LLMs by bringing together three strands of work: early digital methods in HPSS, distributional approaches from digital history and related research, and lexical semantic change detection. We provide an overview of the main challenges and opportunities, focusing on corpus construction, operationalization and modelling choices, and evaluation and interpretation. In the second part, we turn to the era of LLMs, starting with a short introduction to LLMs before reviewing LLM-based work on lexical semantic change detection and relevant case studies in HPSS. We then revisit the earlier methodological questions, showing how issues of corpus construction, model choice and training data, operationalization trade-offs, and evaluation and interpretation play out in LLM-based workflows.