# Beyond Perplexity: Character Distribution Signatures and the MDTA Benchmark for AI Text Detection

arXiv: https://arxiv.org/abs/2605.01647 · HTML: https://mlsanity.com/paper/2605.01647/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 27 references that are in the map.
- First posted on arXiv 3 May 2026.

## Builds on (most cited first, 15 of 15)

- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [DNA-GPT: Divergent N-Gram Analysis for Training-Free Detection of GPT-Generated Text](https://mlsanity.com/paper/2305.17359/) (2023)
- [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023)
- [Ghostbuster: Detecting Text Ghostwritten by Large Language Models](https://mlsanity.com/paper/2305.15047/) (2023)
- [Stylometric Detection of AI-Generated Text in Twitter Timelines](https://mlsanity.com/paper/2303.03697/) (2023)
- [DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios](https://mlsanity.com/paper/2410.23746/) (2024)
- [Unsupervised and Distributional Detection of Machine-Generated Text](https://mlsanity.com/paper/2111.02878/) (2021)
- [DNA-DetectLLM: Unveiling AI-Generated Text via a DNA-Inspired Mutation-Repair Paradigm](https://mlsanity.com/paper/2509.15550/) (2025)
- [Reliably Bounding False Positives: A Zero-Shot Machine-Generated Text Detection Framework via Multiscaled Conformal Prediction](https://mlsanity.com/paper/2505.05084/) (2025)
- [AI-Generated Text Detection and Classification Based on BERT Deep Learning Algorithm](https://mlsanity.com/paper/2405.16422/) (2024)

## Abstract (arXiv)

Training-free AI text detection methods primarily rely on model log-probabilities, achieving strong performance through approaches like Binoculars and DNA-DetectLLM. However, these methods face a fundamental ceiling as models are optimized through RLHF to produce human-like probability distributions. We introduce an alternative detection signal based on character distribution signatures. We provide theoretical foundations showing that AI models, trained on massive domain-balanced corpora, approximate global character patterns while humans exhibit domain-specialized distributions, creating a "Wall of Separation" where human-AI divergence significantly exceeds AI-AI divergence. To enable systematic evaluation, we construct the Models-Domains-Temperatures-Adversarials (MDTA) benchmark comprising 642,274 prompt-aligned samples across 4 models, 5 domains, 3 temperature settings, and 3 adversarial strategies, substantially expanding the HC3 dataset with modern model responses, temperature variation, and adversarial augmentation. We introduce the Letter Distribution Score (LD-Score), demonstrating low correlation (r = 0.08-0.13) with perplexity methods. When integrated with DNA-DetectLLM, Binoculars and FastDetectGPT via a non-linear classifier, LD-Score yields consistent improvements in AUROC and F1, with particularly pronounced gains in specialized domains where vocabulary constraints amplify the detection signal. The MDTA dataset can be accessed at: https://huggingface.co/datasets/nsp909/MDTA.