# Identifying Emerging Concepts in Large Corpora

arXiv: https://arxiv.org/abs/2502.21315 · HTML: https://mlsanity.com/paper/2502.21315/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 43 references that are in the map.
- First posted on arXiv 28 Feb 2025.

## Builds on (most cited first, 16 of 16)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Diachronic Word Embeddings Reveal Statistical Laws of Semantic Change](https://mlsanity.com/paper/1605.09096/) (2016)
- [Top2Vec: Distributed Representations of Topics](https://mlsanity.com/paper/2008.09470/) (2020)
- [Analysing Lexical Semantic Change with Contextualised Word Representations](https://mlsanity.com/paper/2004.14118/) (2020)
- [Statistically Significant Detection of Linguistic Change](https://mlsanity.com/paper/1411.3315/) (2014)
- [Tired of Topic Models? Clusters of Pretrained Word Embeddings Make for Fast and Good Topics too!](https://mlsanity.com/paper/2004.14914/) (2020)
- [Leveraging Contextual Embeddings for Detecting Diachronic Semantic Shift](https://mlsanity.com/paper/1912.01072/) (2019)
- [Short-Term Meaning Shift: A Distributional Exploration](https://mlsanity.com/paper/1809.03169/) (2018)
- [A Survey on Contextualised Semantic Shift Detection](https://mlsanity.com/paper/2304.01666/) (2023)
- [The Diversity-Innovation Paradox in Science](https://mlsanity.com/paper/1909.02063/) (2019)
- [Making "fetch" happen: The influence of social and linguistic context on nonstandard word growth and decline](https://mlsanity.com/paper/1802.04140/) (2018)
- [Specious Sites: Tracking the Spread and Sway of Spurious News Stories at Scale](https://mlsanity.com/paper/2308.02068/) (2023)

## Built on by (most cited first, 1 of 1)

- [HistLens: Mapping Idea Change across Concepts and Corpora](https://mlsanity.com/paper/2604.11749/) (2026)

## Abstract (arXiv)

We introduce a new method to identify emerging concepts in large text corpora. By analyzing changes in the heatmaps of the underlying embedding space, we are able to detect these concepts with high accuracy shortly after they originate, in turn outperforming common alternatives. We further demonstrate the utility of our approach by analyzing speeches in the U.S. Senate from 1941 to 2015. Our results suggest that the minority party is more active in introducing new concepts into the Senate discourse. We also identify specific concepts that closely correlate with the Senators' racial, ethnic, and gender identities. An implementation of our method is publicly available.