# Attribution Quality in AI-Generated Content:Benchmarking Style Embeddings and LLM Judges

arXiv: https://arxiv.org/abs/2510.13898 · HTML: https://mlsanity.com/paper/2510.13898/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 21 references that are in the map.
- First posted on arXiv 14 Oct 2025.

## Builds on (most cited first, 7 of 7)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [Same Author or Just Same Topic? Towards Content-Independent Style Representations](https://mlsanity.com/paper/2204.04907/) (2022)
- [Do LLMs write like humans? Variation in grammatical and rhetorical styles](https://mlsanity.com/paper/2410.16107/) (2024)
- [IMPersona: Evaluating Individual Level LM Impersonation](https://mlsanity.com/paper/2504.04332/) (2025)

## Built on by (most cited first, 1 of 1)

- [Large Language Models and Forensic Linguistics: Navigating Opportunities and Threats in the Age of Generative AI](https://mlsanity.com/paper/2512.06922/) (2025)

## Abstract (arXiv)

Attributing authorship in the era of large language models (LLMs) is increasingly challenging as machine-generated prose rivals human writing. We benchmark two complementary attribution mechanisms , fixed Style Embeddings and an instruction-tuned LLM judge (GPT-4o) on the Human AI Parallel Corpus, an open dataset of 600 balanced instances spanning six domains (academic, news, fiction, blogs, spoken transcripts, and TV/movie scripts). Each instance contains a human prompt with both a gold continuation and an LLM-generated continuation from either GPT-4o or LLaMA-70B-Instruct. The Style Embedding baseline achieves stronger aggregate accuracy on GPT continuations (82 pct vs. 68 pct). The LLM Judge is slightly better than the Style embeddings on LLaMA continuations (85 pct vs. 81 pct) but the results are not statistically significant. Crucially, the LLM judge significantly outperforms in fiction and academic prose, indicating semantic sensitivity, whereas embeddings dominate in spoken and scripted dialogue, reflecting structural strengths. These complementary patterns highlight attribution as a multidimensional problem requiring hybrid strategies. To support reproducibility we provide code on GitHub and derived data on Hugging Face under the MIT license. This open framework provides a reproducible benchmark for attribution quality assessment in AI-generated content, along with a review of related literature influencing this work.