# Community-Specific Slang and Entity Detection via Semantic Shift in Fine-Tuned Language Models

arXiv: https://arxiv.org/abs/2606.07522 · HTML: https://mlsanity.com/paper/2606.07522/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 5 of its 9 references that are in the map.
- First posted on arXiv 19 Apr 2026.

## Builds on (most cited first, 5 of 5)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Mitigating Racial Biases in Toxic Language Detection with an Equity-Based Ensemble Framework](https://mlsanity.com/paper/2109.13137/) (2021)
- [Statistically significant detection of semantic shifts using contextual word embeddings](https://mlsanity.com/paper/2104.03776/) (2021)
- [Semantic Variation in Online Communities of Practice](https://mlsanity.com/paper/1806.05847/) (2018)
- [Information flow reveals prediction limits in online social activity](https://mlsanity.com/paper/1708.04575/) (2017)

## Abstract (arXiv)

We propose an unsupervised method of resolving slang, unique entities, and folklore from online communities by isolating words in the lexicon that have the highest magnitude of semantic shift. Semantic shift is defined as the evolution of a word's encoded representation as a result of fine-tuning a pretrained Large Language Model (LLM) on a community-specific text corpus. This value is inversely proportional to the cosine similarity between the base model's encoded representation of a word, and a fine-tuned model's encoded representation. We fine-tune the DistilRoBERTa model on text corpora collected from 3 Reddit subreddits (r/Technology, r/Gaming, r/WorldofWarcraft), model a distribution of cosine similarity over the lexicon, and show that one can successfully resolve words that have unique significance to the community by pulling data in the bottom 10-percentile. In contrast, we show that data in the top 10-percentile consist of words that carry relatively universal semantics.