# Layered Insights: Generalizable Analysis of Authorial Style by Leveraging All Transformer Layers

arXiv: https://arxiv.org/abs/2503.00958 · HTML: https://mlsanity.com/paper/2503.00958/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 30 references that are in the map.
- First posted on arXiv 2 Mar 2025.

## Builds on (most cited first, 14 of 14)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://mlsanity.com/paper/2412.13663/) (2024)
- [Self-attention Does Not Need O(n2) Memory](https://mlsanity.com/paper/2112.05682/) (2021)
- [Not All Layers of LLMs Are Necessary During Inference](https://mlsanity.com/paper/2403.02181/) (2024)
- [Same Author or Just Same Topic? Towards Content-Independent Style Representations](https://mlsanity.com/paper/2204.04907/) (2022)
- [How Does BERT Answer Questions? A Layer-Wise Analysis of Transformer Representations](https://mlsanity.com/paper/1909.04925/) (2019)
- [Holmes: A Benchmark to Assess the Linguistic Competence of Language Models](https://mlsanity.com/paper/2404.18923/) (2024)
- [A Deep Metric Learning Approach to Account Linking](https://mlsanity.com/paper/2105.07263/) (2021)
- [Learning Invariant Representations of Social Media Users](https://mlsanity.com/paper/1910.04979/) (2019)
- [The Importance of Suppressing Domain Style in Authorship Analysis](https://mlsanity.com/paper/2005.14714/) (2020)

## Built on by (most cited first, 2 of 2)

- [Where Does Authorship Signal Emerge in Encoder-Based Language Models?](https://mlsanity.com/paper/2605.19908/) (2026)
- [HyperStyler: Low-resource Authorship Style Transfer via Context-aware Style Navigation and Hypernetworks](https://mlsanity.com/paper/2609.02772/) (2026)

## Abstract (arXiv)

We propose a new approach for the authorship attribution task that leverages the various linguistic representations learned at different layers of pre-trained transformer-based models. We evaluate our approach on three datasets, comparing it to a state-of-the-art baseline in in-domain and out-of-domain scenarios. We found that utilizing various transformer layers improves the robustness of authorship attribution models when tested on out-of-domain data, resulting in new state-of-the-art results. Our analysis gives further insights into how our model's different layers get specialized in representing certain stylistic features that benefit the model when tested out of the domain.