# Capturing Style in Author and Document Representation

arXiv: https://arxiv.org/abs/2407.13358 · HTML: https://mlsanity.com/paper/2407.13358/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 40 references that are in the map.
- First posted on arXiv 18 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Big Bird: Transformers for Longer Sequences](https://mlsanity.com/paper/2007.14062/) (2020)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Is ChatGPT a General-Purpose Natural Language Processing Task Solver?](https://mlsanity.com/paper/2302.06476/) (2023)
- [A standardized Project Gutenberg corpus for statistical analysis of natural language and quantitative linguistics](https://mlsanity.com/paper/1812.08092/) (2018)
- [Modeling Uncertainty with Hedged Instance Embedding](https://mlsanity.com/paper/1810.00319/) (2018)
- [Variational Information Bottleneck for Effective Low-Resource Fine-Tuning](https://mlsanity.com/paper/2106.05469/) (2021)
- [Character-level and Multi-channel Convolutional Neural Networks for Large-scale Authorship Attribution](https://mlsanity.com/paper/1609.06686/) (2016)
- [Quantifying Mental Health from Social Media with Neural User Embeddings](https://mlsanity.com/paper/1705.00335/) (2017)
- [Style-aware Neural Model with Application in Authorship Attribution](https://mlsanity.com/paper/1909.06194/) (2019)

## Built on by (most cited first, 1 of 1)

- [Author Representation Strategies for Zero-Shot Authorship Attribution: A Comparative Study of LLM-Based and Embedding-Based Approaches](https://mlsanity.com/paper/2610.03531/) (2026)

## Abstract (arXiv)

A wide range of Deep Natural Language Processing (NLP) models integrates continuous and low dimensional representations of words and documents. Surprisingly, very few models study representation learning for authors. These representations can be used for many NLP tasks, such as author identification and classification, or in recommendation systems. A strong limitation of existing works is that they do not explicitly capture writing style, making them hardly applicable to literary data. We therefore propose a new architecture based on Variational Information Bottleneck (VIB) that learns embeddings for both authors and documents with a stylistic constraint. Our model fine-tunes a pre-trained document encoder. We stimulate the detection of writing style by adding predefined stylistic features making the representation axis interpretable with respect to writing style indicators. We evaluate our method on three datasets: a literary corpus extracted from the Gutenberg Project, the Blog Authorship Corpus and IMDb62, for which we show that it matches or outperforms strong/recent baselines in authorship attribution while capturing much more accurately the authors stylistic aspects.