# StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples

arXiv: https://arxiv.org/abs/2410.12757 · HTML: https://mlsanity.com/paper/2410.12757/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 56 references that are in the map.
- First posted on arXiv 16 Oct 2024.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [Datasets: A Community Library for Natural Language Processing](https://mlsanity.com/paper/2109.02846/) (2021)
- [Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias](https://mlsanity.com/paper/2306.15895/) (2023)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Deep Learning for Text Style Transfer: A Survey](https://mlsanity.com/paper/2011.00416/) (2020)
- [Same Author or Just Same Topic? Towards Content-Independent Style Representations](https://mlsanity.com/paper/2204.04907/) (2022)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [Prompt-and-Rerank: A Method for Zero-Shot and Few-Shot Arbitrary Textual Style Transfer with Small Language Models](https://mlsanity.com/paper/2205.11503/) (2022)
- [DataDreamer: A Tool for Synthetic Data Generation and Reproducible LLM Workflows](https://mlsanity.com/paper/2402.10379/) (2024)
- [TextSETTR: Few-Shot Text Style Extraction and Tunable Targeted Restyling](https://mlsanity.com/paper/2010.03802/) (2020)
- [Learning Interpretable Style Embeddings via Prompting LLMs](https://mlsanity.com/paper/2305.12696/) (2023)
- [A Deep Metric Learning Approach to Account Linking](https://mlsanity.com/paper/2105.07263/) (2021)
- [ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer](https://mlsanity.com/paper/2308.15459/) (2023)
- [StyleRemix: Interpretable Authorship Obfuscation via Distillation and Perturbation of Style Elements](https://mlsanity.com/paper/2408.15666/) (2024)
- [Does It Capture STEL? A Modular, Similarity-based Linguistic Style Evaluation Framework](https://mlsanity.com/paper/2109.04817/) (2021)
- [A Textbook Remedy for Domain Shifts: Knowledge Priors for Medical Image Analysis](https://mlsanity.com/paper/2405.14839/) (2024)
- [Authorship Style Transfer with Policy Optimization](https://mlsanity.com/paper/2403.08043/) (2024)
- [Learning to Generate Text in Arbitrary Writing Styles](https://mlsanity.com/paper/2312.17242/) (2023)
- [StyleFlow: Disentangle Latent Representations via Normalizing Flow for Unsupervised Text Style Transfer](https://mlsanity.com/paper/2212.09670/) (2022)

## Built on by (most cited first, 30 of 35)

- [Leveraging Multilingual Training for Authorship Representation: Enhancing Generalization across Languages and Domains](https://mlsanity.com/paper/2509.16531/) (2025)
- [Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Models](https://mlsanity.com/paper/2507.13614/) (2025)
- [Evaluating Style-Personalized Text Generation: Challenges and Directions](https://mlsanity.com/paper/2508.06374/) (2025)
- [NSF-SciFy: Mining the NSF Awards Database for Scientific Claims](https://mlsanity.com/paper/2503.08600/) (2025)
- [The Impact of Automatic Speech Transcription on Speaker Attribution](https://mlsanity.com/paper/2507.08660/) (2025)
- [Multiple Choice Learning of Low-Rank Adapters for Language Modeling](https://mlsanity.com/paper/2507.10419/) (2025)
- [A stylometric analysis of speaker attribution from speech transcripts](https://mlsanity.com/paper/2512.13667/) (2025)
- [Benchmarking Machine Translation on Chinese Social Media Texts](https://mlsanity.com/paper/2601.22931/) (2026)
- [IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variation](https://mlsanity.com/paper/2604.04704/) (2026)
- [mStyleDistance: Multilingual Style Embeddings and their Evaluation](https://mlsanity.com/paper/2502.15168/) (2025)
- [Attacks on Machine-Text Detectors Retain Stylistic Fingerprints](https://mlsanity.com/paper/2505.14608/) (2025)
- [Team "better_call_claude": Style Change Detection using a Sequential Sentence Pair Classifier](https://mlsanity.com/paper/2508.00675/) (2025)
- [One-shot Style Transfer LLM log-probabilities for Authorship Attribution and Verification](https://mlsanity.com/paper/2510.13302/) (2025)
- [CLASE: A Hybrid Method for Chinese Legalese Stylistic Evaluation](https://mlsanity.com/paper/2602.12639/) (2026)
- [AuthorMix: Modular Authorship Style Transfer via Layer-wise Adapter Mixing](https://mlsanity.com/paper/2603.23069/) (2026)
- [Interpretable Stylistic Variation in Human and LLM Writing Across Genres, Models, and Decoding Strategies](https://mlsanity.com/paper/2604.14111/) (2026)
- [Can You Make It Sound Like You? Post-Editing LLM-Generated Text for Personal Style](https://mlsanity.com/paper/2604.24444/) (2026)
- [Narrative Flattening: How Post-Training Compresses Thematic, Affective, and Stylistic Variation in LLM Fiction](https://mlsanity.com/paper/2605.27878/) (2026)
- [Challenging Assumptions in Learning Generic Text Style Embeddings](https://mlsanity.com/paper/2501.16073/) (2025)
- [Quantifying Misattribution Unfairness in Authorship Attribution](https://mlsanity.com/paper/2506.02321/) (2025)
- [Computational Phenomenology of Borderline Personality Disorder: A Comparative Evaluation of LLM-Simulated Expert Personas and Human Clinical Experts](https://mlsanity.com/paper/2508.19008/) (2025)
- [RL-Finetuned LLMs for Privacy-Preserving Synthetic Rewriting](https://mlsanity.com/paper/2508.19286/) (2025)
- [Navigating the Synchrony-Stability Frontier in Adaptive Chatbots](https://mlsanity.com/paper/2510.00339/) (2025)
- [iBERT: Interpretable Embeddings via Sense Decomposition](https://mlsanity.com/paper/2510.09882/) (2025)
- [Capturing Classic Authorial Style in Long-Form Story Generation with GRPO Fine-Tuning](https://mlsanity.com/paper/2512.05747/) (2025)
- [Interpreting Style Representations via Style-Eliciting Prompts](https://mlsanity.com/paper/2606.05716/) (2026)
- [Unsupervised Style Representation Learning for AI-Text Detection via Paraphrase Inversion](https://mlsanity.com/paper/2606.10099/) (2026)
- [STEB: Style Text Embedding Benchmark](https://mlsanity.com/paper/2606.31741/) (2026)
- [Writing Style Similarity Reflects Academic Genealogy](https://mlsanity.com/paper/2608.14843/) (2026)
- [Synthetic Linguistic Agency: How an Embodied Mortal Agent Learns Linguistic Affordances through Consequential Social Experience](https://mlsanity.com/paper/2608.27843/) (2026)

## Abstract (arXiv)

Style representations aim to embed texts with similar writing styles closely and texts with different styles far apart, regardless of content. However, the contrastive triplets often used for training these representations may vary in both style and content, leading to potential content leakage in the representations. We introduce StyleDistance, a novel approach to training stronger content-independent style embeddings. We use a large language model to create a synthetic dataset of near-exact paraphrases with controlled style variations, and produce positive and negative examples across 40 distinct style features for precise contrastive learning. We assess the quality of our synthetic data and embeddings through human and automatic evaluations. StyleDistance enhances the content-independence of style embeddings, which generalize to real-world benchmarks and outperform leading style representations in downstream applications. Our model can be found at https://huggingface.co/StyleDistance/styledistance .