# Rethinking the Authorship Verification Experimental Setups

arXiv: https://arxiv.org/abs/2112.05125 · HTML: https://mlsanity.com/paper/2112.05125/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 9 Dec 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Captum: A unified and generic model interpretability library for PyTorch](https://mlsanity.com/paper/2009.07896/) (2020)
- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Is Attention Interpretable?](https://mlsanity.com/paper/1906.03731/) (2019)
- [The elephant in the interpretability room: Why use attention as explanation when we have saliency methods?](https://mlsanity.com/paper/2010.05607/) (2020)
- [Learning to Deceive with Attention-Based Explanations](https://mlsanity.com/paper/1909.07913/) (2019)
- [CharacterBERT: Reconciling ELMo and BERT for Word-Level Open-Vocabulary Representations From Characters](https://mlsanity.com/paper/2010.10392/) (2020)
- [Siamese Networks for Large-Scale Author Identification](https://mlsanity.com/paper/1912.10616/) (2019)
- [O2D2: Out-Of-Distribution Detector to Capture Undecidable Trials in Authorship Verification](https://mlsanity.com/paper/2106.15825/) (2021)

## Built on by (most cited first, 5 of 5)

- [InstructAV: Instruction Fine-tuning Large Language Models for Authorship Verification](https://mlsanity.com/paper/2407.12882/) (2024)
- [Trends and Challenges in Authorship Analysis: A Review of ML, DL, and LLM Approaches](https://mlsanity.com/paper/2505.15422/) (2025)
- [Addressing Topic Leakage in Cross-Topic Evaluation for Authorship Verification](https://mlsanity.com/paper/2407.19164/) (2024)
- [Authorship identification under domain shift: a survey of stylistic measures and learned author representations](https://mlsanity.com/paper/2310.00436/) (2023)
- [Large Language Models Threaten Double-blind Review](https://mlsanity.com/paper/2608.05157/) (2026)

## Abstract (arXiv)

One of the main drivers of the recent advances in authorship verification is the PAN large-scale authorship dataset. Despite generating significant progress in the field, inconsistent performance differences between the closed and open test sets have been reported. To this end, we improve the experimental setup by proposing five new public splits over the PAN dataset, specifically designed to isolate and identify biases related to the text topic and to the author's writing style. We evaluate several BERT-like baselines on these splits, showing that such models are competitive with authorship verification state-of-the-art methods. Furthermore, using explainable AI, we find that these baselines are biased towards named entities. We show that models trained without the named entities obtain better results and generalize better when tested on DarkReddit, our new dataset for authorship verification.