Where Does Authorship Signal Emerge in Encoder-Based Language Models?
Not on the map. Why?
Abstract (arXiv)
Authorship attribution models fine-tuned with the same pretrained encoder, data, and loss can differ four-fold in performance depending only on their scoring mechanism. We use mechanistic interpretability tools to explain this gap. Stylistic features such as word length, punctuation density, and function-word frequency are similarly available at every layer in every model we probe, including an off-the-shelf control encoder, suggesting that the gap is not explained by their linear readability. Instead, causal intervention shows that the scorer appears to determine where the encoder consolidates authorship signal. Mean pooling forces consolidation by early to mid layers, while late interaction defers it to later layers. We further derive this difference from the gradient structure of each scorer, and training dynamics reveal distinct learning trajectories that follow from that difference.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 25 of its 36 references that are in the map. First posted on arXiv 19 May 2026.
Builds on
25 of 36 references.
References · 25
- 2017Attention Is All You Needcited by 60,361
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
- 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
- 2022Locating and Editing Factual Associations in GPTcited by 2,691
- 2020Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hyperspherecited by 1,405
- 2016Understanding intermediate layers using linear classifier probescited by 1,353
- 2022Text Embeddings by Weakly-Supervised Contrastive Pre-trainingcited by 1,253
- 2020ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERTcited by 1,178
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
13 more
- 2019Designing and Interpreting Probes with Control Taskscited by 640
- 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
- 2023Localizing Model Behavior with Path Patchingcited by 195
- 2020Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?cited by 151
- 2022Same Author or Just Same Topic? Towards Content-Independent Style Representationscited by 86
- 2021Does It Capture STEL? A Modular, Similarity-based Linguistic Style Evaluation Frameworkcited by 17
- 2022Whodunit? Learning to Contrast for Authorship Attributioncited by 17
- 2020Why Moli`ere most likely did write his playscited by 9
- 2017Git Blame Who?: Stylistic Authorship Attribution of Small, Incomplete Source Code Fragmentscited by 7
- 2024Latent Space Interpretation for Stylistic Analysis and Explainable Authorship Attributioncited by 7
- 2025Layered Insights: Generalizable Analysis of Authorial Style by Leveraging All Transformer Layerscited by 2
- 2026IDIOLEX: Unified and Continuous Representations for Idiolectal and Stylistic Variationcited by 2
- 2024HALvest-Contrastive: Retrieval-Like Authorship Attribution with Patch-Level Late Interactioncited by 1
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Can Large Language Models Identify Authorship?
- 2026Can Activation Steering Capture Multidimensional Authorship Style?
- 2025On Support Samples of Next Word Prediction
- 2026Measuring, Localizing, and Ablating Alignment Signatures in LLMs
- 2024A Bayesian Approach to Harnessing the Power of LLMs in Authorship Attribution
- 2024Detecting Mode Collapse in Language Models via Narration
- 2026When Attribution Patching Lies: Diagnosis and a Second-Order Correction
- 2026Concept Influence: Leveraging Interpretability to Improve Performance and Efficiency in Training Data Attribution