# Embedding Style Beyond Topics: Analyzing Dispersion Effects Across Different Language Models

arXiv: https://arxiv.org/abs/2501.00828 · HTML: https://mlsanity.com/paper/2501.00828/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 42 references that are in the map.
- First posted on arXiv 1 Jan 2025.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [BERTopic: Neural topic modeling with a class-based TF-IDF procedure](https://mlsanity.com/paper/2203.05794/) (2022)
- [Explainability for Large Language Models: A Survey](https://mlsanity.com/paper/2309.01029/) (2023)
- [Topic Modeling in Embedding Spaces](https://mlsanity.com/paper/1907.04907/) (2019)
- [A Tutorial on Principal Component Analysis](https://mlsanity.com/paper/1404.1100/) (2014)
- [Self-Attention Attribution: Interpreting Information Interactions Inside Transformer](https://mlsanity.com/paper/2004.11207/) (2020)
- [Same Author or Just Same Topic? Towards Content-Independent Style Representations](https://mlsanity.com/paper/2204.04907/) (2022)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [Directed Diversity: Leveraging Language Embedding Distances for Collective Creativity in Crowd Ideation](https://mlsanity.com/paper/2101.06030/) (2021)
- [A Lexical, Syntactic, and Semantic Perspective for Understanding Style in Text](https://mlsanity.com/paper/1909.08349/) (2019)
- [A Multi-Label Dataset of French Fake News: Human and Machine Insights](https://mlsanity.com/paper/2403.16099/) (2024)
- [Exposing propaganda: an analysis of stylistic cues comparing human annotations and machine classification](https://mlsanity.com/paper/2402.03780/) (2024)

## Built on by (most cited first, 1 of 1)

- [Spacer: Towards Engineered Scientific Inspiration](https://mlsanity.com/paper/2508.17661/) (2025)

## Abstract (arXiv)

This paper analyzes how writing style affects the dispersion of embedding vectors across multiple, state-of-the-art language models. While early transformer models primarily aligned with topic modeling, this study examines the role of writing style in shaping embedding spaces. Using a literary corpus that alternates between topics and styles, we compare the sensitivity of language models across French and English. By analyzing the particular impact of style on embedding dispersion, we aim to better understand how language models process stylistic information, contributing to their overall interpretability.