# ARB: A Matched Authorship-Rewriting Benchmark Dataset for AI-Text Detector Evaluation

arXiv: https://arxiv.org/abs/2607.29539 · HTML: https://mlsanity.com/paper/2607.29539/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 53 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Builds on (most cited first, 30 of 38)

- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [Large Language Models Are Human-Level Prompt Engineers](https://mlsanity.com/paper/2211.01910/) (2022)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applications](https://mlsanity.com/paper/2402.07927/) (2024)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond](https://mlsanity.com/paper/2304.13712/) (2023)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://mlsanity.com/paper/2405.07940/) (2024)
- [MAGE: Machine-generated Text Detection in the Wild](https://mlsanity.com/paper/2305.13242/) (2023)
- [Increasing Diversity While Maintaining Accuracy: Text Data Generation with Large Language Models and Human Interventions](https://mlsanity.com/paper/2306.04140/) (2023)
- [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023)
- [Testing of Detection Tools for AI-Generated Text](https://mlsanity.com/paper/2306.15666/) (2023)
- [Red Teaming Language Model Detectors with Language Models](https://mlsanity.com/paper/2305.19713/) (2023)
- [Raidar: geneRative AI Detection viA Rewriting](https://mlsanity.com/paper/2401.12970/) (2024)
- [M4GT-Bench: Evaluation Benchmark for Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2402.11175/) (2024)
- [DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios](https://mlsanity.com/paper/2410.23746/) (2024)
- [Deepfake Text Detection: Limitations and Opportunities](https://mlsanity.com/paper/2210.09421/) (2022)
- [Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore](https://mlsanity.com/paper/2405.04286/) (2024)
- [DAMAGE: Detecting Adversarially Modified AI Generated Text](https://mlsanity.com/paper/2501.03437/) (2025)
- [Balancing Diversity and Risk in LLM Sampling: How to Select Your Method and Parameter for Open-Ended Text Generation](https://mlsanity.com/paper/2408.13586/) (2024)
- [Learning to Rewrite: Generalized LLM-Generated Text Detection](https://mlsanity.com/paper/2408.04237/) (2024)
- [DivScore: Zero-Shot Detection of LLM-Generated Text in Specialized Domains](https://mlsanity.com/paper/2506.06705/) (2025)
- [Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors](https://mlsanity.com/paper/2410.19230/) (2024)
- [Understanding the Effects of Human-written Paraphrases in LLM-generated Text Detection](https://mlsanity.com/paper/2411.03806/) (2024)
- [StealthRL: Reinforcement Learning Paraphrase Attacks for Multi-Detector Evasion of AI-Text Detectors](https://mlsanity.com/paper/2602.08934/) (2026)
- [Beyond Easy Wins: A Text Hardness-Aware Benchmark for LLM-generated Text Detection](https://mlsanity.com/paper/2507.15286/) (2025)

## Built on by (most cited first, 1 of 1)

- [Forging LLM Authorship Fingerprints with Targeted Rewriting](https://mlsanity.com/paper/2609.38831/) (2026)

## Abstract (arXiv)

Standard AI-text detection benchmarks compare human-written text against text generated directly by large language models (LLMs). While prior work has shown that rewriting and paraphrasing can degrade detector performance, it remains unclear whether performance measured on this conventional benchmark predicts detector behavior when human-authored content is rewritten by an LLM. To address this gap, we introduce Authorship-Rewriting Benchmark (ARB), built from 1,800 human source texts (600 each from XSum, WritingPrompts, and OpenWebText) and four open-weight generators (Llama-3.2-3B, Qwen2.5-7B, Mistral-7B, Gemma-2-9B). Each source item yields four matched variants: human-written (HUMAN), direct LLM generation (Free-LLM), LLM-rewritten human text (H2L), and same-generator LLM-rewritten LLM text (LLM2L). We evaluated five detectors (FastDetectGPT, Binoculars-falcon-7b, RADAR, BERT-Defense, RoBERTa-Defense) at a strict 1%-false-positive operating point (TPR@1%FPR). FastDetectGPT and Binoculars-falcon-7b detected 91.2% and 93.5% of direct LLM text, but only 30.8% and 15.1% of human text an LLM had rewritten, a drop of 60-78 percentage points. The same detectors retained 78.3% and 83.0% recall when LLM text was rewritten by the same model, a much smaller decline of 10-13 points. RADAR followed the same pattern (66.8% to 12.2%), while BERT-Defense and RoBERTa-Defense stayed below 3% recall across all regimes. These results show that detector performance measured on the conventional human-vs-LLM benchmark does not transfer to human-authored text revised by an LLM, even though the same detectors remain largely robust to LLM-only rewriting.