# Benchmark of stylistic variation in LLM-generated texts

arXiv: https://arxiv.org/abs/2509.10179 · HTML: https://mlsanity.com/paper/2509.10179/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 43 references that are in the map.
- First posted on arXiv 12 Sep 2025.

## Builds on (most cited first, 13 of 13)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Constitutional AI: Harmlessness from AI Feedback](https://mlsanity.com/paper/2212.08073/) (2022)
- [Role-Play with Large Language Models](https://mlsanity.com/paper/2305.16367/) (2023)
- [Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMs](https://mlsanity.com/paper/2402.03927/) (2024)
- [The Ghost in the Machine has an American accent: value conflict in GPT-3](https://mlsanity.com/paper/2203.07785/) (2022)
- [The Political Preferences of LLMs](https://mlsanity.com/paper/2402.01789/) (2024)
- [Do LLMs write like humans? Variation in grammatical and rhetorical styles](https://mlsanity.com/paper/2410.16107/) (2024)
- [Stylometric Detection of AI-Generated Text in Twitter Timelines](https://mlsanity.com/paper/2303.03697/) (2023)
- [Beyond English-Centric LLMs: What Language Do Multilingual Language Models Think in?](https://mlsanity.com/paper/2408.10811/) (2024)
- [Training on the Benchmark Is Not All You Need](https://mlsanity.com/paper/2409.01790/) (2024)
- [Detecting Stylistic Fingerprints of Large Language Models](https://mlsanity.com/paper/2503.01659/) (2025)
- [Humans can learn to detect AI-generated texts, or at least learn when they can't](https://mlsanity.com/paper/2505.01877/) (2025)

## Built on by (most cited first, 7 of 7)

- [AI Brown and AI Koditex: LLM-Generated Corpora Comparable to Traditional Corpora of English and Czech Texts](https://mlsanity.com/paper/2509.22996/) (2025)
- [The Rise of Verbal Tics in Large Language Models: A Systematic Analysis Across Frontier Models](https://mlsanity.com/paper/2604.19139/) (2026)
- [How Human-Like Are Large Language Models? A Register-Aware Linguistic Evaluation Framework](https://mlsanity.com/paper/2605.23651/) (2026)
- [From Context Shift to Stylistic Collapse: Why Training Objectives Matter More Than Scale](https://mlsanity.com/paper/2605.28826/) (2026)
- [Production and Perception in LLMs: A Token Probability Approach](https://mlsanity.com/paper/2607.11703/) (2026)
- [Optimization Is Not All You Need](https://mlsanity.com/paper/2607.11977/) (2026)
- [Beyond Fluency: A Rubric-Based Benchmark for Evaluating Saudi Dialect and Cultural Competence in Large Language Models](https://mlsanity.com/paper/2608.29990/) (2026)

## Abstract (arXiv)

This study investigates the register variation in texts written by humans and comparable texts produced by large language models (LLMs). Biber's multidimensional analysis (MDA) is applied to a sample of human-written texts and AI-created texts generated to be their counterparts to find the dimensions of variation in which LLMs differ most significantly and most systematically from humans. As textual material, a new LLM-generated corpus AI-Brown is used, which is comparable to BE-21 (a Brown family corpus representing contemporary British English). Since all languages except English are underrepresented in the training data of frontier LLMs, similar analysis is replicated on Czech using AI-Koditex corpus and Czech multidimensional model. Examined were 16 frontier models in various settings and prompts, with emphasis placed on the difference between base models and instruction-tuned models. Based on this, a benchmark is created through which models can be compared with each other and ranked in interpretable dimensions.