Better Call Claude: Can LLMs Detect Changes of Writing Style?
Not on the map. Why?
Abstract (arXiv)
This article explores the zero-shot performance of state-of-the-art large language models (LLMs) on one of the most challenging tasks in authorship analysis: sentence-level style change detection. Benchmarking four LLMs on the official PAN~2024 and 2025 "Multi-Author Writing Style Analysis" datasets, we present several observations. First, state-of-the-art generative models are sensitive to variations in writing style - even at the granular level of individual sentences. Second, their accuracy establishes a challenging baseline for the task, outperforming suggested baselines of the PAN competition. Finally, we explore the influence of semantics on model predictions and present evidence suggesting that the latest generation of LLMs may be more sensitive to content-independent and purely stylistic signals than previously reported.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 35 references that are in the map. First posted on arXiv 1 Aug 2025.
Builds on
20 of 35 references.
References · 20
- 2022Training language models to follow instructions with human feedbackcited by 15,712
- 2022Large Language Models are Zero-Shot Reasonerscited by 4,323
- 2020DeBERTa: Decoding-enhanced BERT with Disentangled Attentioncited by 2,157
- 2021Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigmcited by 599
- 2023Enabling Large Language Models to Generate Text with Citationscited by 584
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2021A Survey on Automated Fact-Checkingcited by 376
- 2023A Survey of Large Language Models Attributioncited by 71
- 2023Automatic Evaluation of Attribution by Large Language Modelscited by 70
- 2023SeqXGPT: Sentence-Level AI-Generated Text Detectioncited by 55
- 2021Artificial Text Detection via Examining the Topology of Attention Mapscited by 49
- 2024Authorship Attribution in the Era of LLMs: Problems, Methodologies, and Challengescited by 46
8 more
- 2023Learning Interpretable Style Embeddings via Prompting LLMscited by 29
- 2021Transformer over Pre-trained Transformer for Neural Text Segmentation with Enhanced Topic Coherencecited by 22
- 2024Detecting AI-Generated Sentences in Human-AI Collaborative Hybrid Texts: Challenges, Strategies, and Insightscited by 21
- 2023Who Wrote it and Why? Prompting Large-Language Models for Authorship Verificationcited by 19
- 2023TOPFORMER: Topology-Aware Authorship Attribution of Deepfake Texts with Diverse Writing Stylescited by 10
- 2024Sui Generis: Large Language Models for Authorship Attribution and Verification in Latincited by 8
- 2024CAVE: Controllable Authorship Verification Explanationscited by 3
- 2016Authorship clustering using multi-headed recurrent neural networkscited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.