# Same Author or Just Same Topic? Towards Content-Independent Style Representations

arXiv: https://arxiv.org/abs/2204.04907 · HTML: https://mlsanity.com/paper/2204.04907/

## Key facts

- Cited by 86 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 45 references that are in the map.
- First posted on arXiv 11 Apr 2022.

## Builds on (most cited first, 22 of 22)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference](https://mlsanity.com/paper/1704.05426/) (2017)
- [A large annotated corpus for learning natural language inference](https://mlsanity.com/paper/1508.05326/) (2015)
- [SemEval-2017 Task 1: Semantic Textual Similarity - Multilingual and Cross-lingual Focused Evaluation](https://mlsanity.com/paper/1708.00055/) (2017)
- [Supervised Learning of Universal Sentence Representations from Natural Language Inference Data](https://mlsanity.com/paper/1705.02364/) (2017)
- [Hypothesis Only Baselines in Natural Language Inference](https://mlsanity.com/paper/1805.01042/) (2018)
- [SentEval: An Evaluation Toolkit for Universal Sentence Representations](https://mlsanity.com/paper/1803.05449/) (2018)
- [The Pushshift Reddit Dataset](https://mlsanity.com/paper/2001.08435/) (2020)
- [Dear Sir or Madam, May I introduce the GYAFC Dataset: Corpus, Benchmarks and Metrics for Formality Style Transfer](https://mlsanity.com/paper/1803.06535/) (2018)
- [Controlling Linguistic Style Aspects in Neural Language Generation](https://mlsanity.com/paper/1707.02633/) (2017)
- [ConvoKit: A Toolkit for the Analysis of Conversations](https://mlsanity.com/paper/2005.04246/) (2020)
- [Personalized Machine Translation: Preserving Original Author Traits](https://mlsanity.com/paper/1610.05461/) (2016)
- [Learning Semantic Textual Similarity from Conversations](https://mlsanity.com/paper/1804.07754/) (2018)
- [Explainable Authorship Verification in Social Media via Attention-based Similarity Learning](https://mlsanity.com/paper/1910.08144/) (2019)
- [DialogueCSE: Dialogue-based Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2109.12599/) (2021)
- [The Importance of Suppressing Domain Style in Authorship Analysis](https://mlsanity.com/paper/2005.14714/) (2020)
- [Idiosyncratic but not Arbitrary: Learning Idiolects in Online Registers Reveals Distinctive yet Consistent Individual Styles](https://mlsanity.com/paper/2109.03158/) (2021)
- [Does It Capture STEL? A Modular, Similarity-based Linguistic Style Evaluation Framework](https://mlsanity.com/paper/2109.04817/) (2021)
- [Siamese Networks for Large-Scale Author Identification](https://mlsanity.com/paper/1912.10616/) (2019)
- [Similarity Learning for Authorship Verification in Social Media](https://mlsanity.com/paper/1908.07844/) (2019)
- [DeepStyle: User Style Embedding for Authorship Attribution of Short Texts](https://mlsanity.com/paper/2103.11798/) (2021)

## Built on by (most cited first, 30 of 86)

- [Can Large Language Models Identify Authorship?](https://mlsanity.com/paper/2403.08213/) (2024)
- [Know You First and Be You Better: Modeling Human-Like User Simulators via Implicit Profiles](https://mlsanity.com/paper/2502.18968/) (2025)
- [Few-Shot Detection of Machine-Generated Text using Style Representations](https://mlsanity.com/paper/2401.06712/) (2024)
- [Modifying Large Language Model Post-Training for Diverse Creative Writing](https://mlsanity.com/paper/2503.17126/) (2025)
- [StyleDistance: Stronger Content-Independent Style Embeddings with Synthetic Parallel Examples](https://mlsanity.com/paper/2410.12757/) (2024)
- [The Homogenizing Effect of Large Language Models on Human Expression and Thought](https://mlsanity.com/paper/2508.01491/) (2025)
- [CoGenesis: A Framework Collaborating Large and Small Language Models for Secure Context-Aware Instruction Following](https://mlsanity.com/paper/2403.03129/) (2024)
- [Learning Interpretable Style Embeddings via Prompting LLMs](https://mlsanity.com/paper/2305.12696/) (2023)
- [RECAP: Retrieval-Enhanced Context-Aware Prefix Encoder for Personalized Dialogue Response Generation](https://mlsanity.com/paper/2306.07206/) (2023)
- [A Ship of Theseus: Curious Cases of Paraphrasing in LLM-Generated Texts](https://mlsanity.com/paper/2311.08374/) (2023)
- [ParaGuide: Guided Diffusion Paraphrasers for Plug-and-Play Textual Style Transfer](https://mlsanity.com/paper/2308.15459/) (2023)
- [Can Authorship Representation Learning Capture Stylistic Features?](https://mlsanity.com/paper/2308.11490/) (2023)
- [TinyStyler: Efficient Few-Shot Text Style Transfer with Authorship Embeddings](https://mlsanity.com/paper/2406.15586/) (2024)
- [PART: Pre-trained Authorship Representation Transformer](https://mlsanity.com/paper/2209.15373/) (2022)
- [Tokenization is Sensitive to Language Variation](https://mlsanity.com/paper/2502.15343/) (2025)
- [Understanding writing style in social media with a supervised contrastively pre-trained transformer](https://mlsanity.com/paper/2310.11081/) (2023)
- [VeriDark: A Large-Scale Benchmark for Authorship Verification on the Dark Web](https://mlsanity.com/paper/2207.03477/) (2022)
- [Low-Resource Authorship Style Transfer: Can Non-Famous Authors Be Imitated?](https://mlsanity.com/paper/2212.08986/) (2022)
- [Learning to Generate Text in Arbitrary Writing Styles](https://mlsanity.com/paper/2312.17242/) (2023)
- [Leveraging Multilingual Training for Authorship Representation: Enhancing Generalization across Languages and Domains](https://mlsanity.com/paper/2509.16531/) (2025)
- [Can Authorship Attribution Models Distinguish Speakers in Speech Transcripts?](https://mlsanity.com/paper/2311.07564/) (2023)
- [Customizing Large Language Model Generation Style using Parameter-Efficient Finetuning](https://mlsanity.com/paper/2409.04574/) (2024)
- [Latent Space Interpretation for Stylistic Analysis and Explainable Authorship Attribution](https://mlsanity.com/paper/2409.07072/) (2024)
- [ICLEF: In-Context Learning with Expert Feedback for Explainable Style Transfer](https://mlsanity.com/paper/2309.08583/) (2023)
- [Steering Large Language Models with Register Analysis for Arbitrary Style Transfer](https://mlsanity.com/paper/2505.00679/) (2025)
- [Distinguishing Fictional Voices: a Study of Authorship Verification Models for Quotation Attribution](https://mlsanity.com/paper/2401.16968/) (2024)
- [Grammar as a Behavioral Biometric: Using Cognitively Motivated Grammar Models for Authorship Verification](https://mlsanity.com/paper/2403.08462/) (2024)
- [Speaker Verification in Agent-Generated Conversations](https://mlsanity.com/paper/2405.10150/) (2024)
- [Decoding Human Preferences in Alignment: An Improved Approach to Inverse Constitutional AI](https://mlsanity.com/paper/2501.17112/) (2025)
- [Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning](https://mlsanity.com/paper/2502.08972/) (2025)

## Abstract (arXiv)

Linguistic style is an integral component of language. Recent advances in the development of style representations have increasingly used training objectives from authorship verification (AV): Do two texts have the same author? The assumption underlying the AV training task (same author approximates same writing style) enables self-supervised and, thus, extensive training. However, a good performance on the AV task does not ensure good "general-purpose" style representations. For example, as the same author might typically write about certain topics, representations trained on AV might also encode content information instead of style alone. We introduce a variation of the AV training task that controls for content using conversation or domain labels. We evaluate whether known style dimensions are represented and preferred over content information through an original variation to the recently proposed STEL framework. We find that representations trained by controlling for conversation are better than representations trained with domain or no content control at representing style independent from content.