# TempTest: Local Normalization Distortion and the Detection of Machine-generated Text

arXiv: https://arxiv.org/abs/2503.20421 · HTML: https://mlsanity.com/paper/2503.20421/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 26 Mar 2025.

## Builds on (most cited first, 27 of 27)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [PubMedQA: A Dataset for Biomedical Research Question Answering](https://mlsanity.com/paper/1909.06146/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023)
- [DetectLLM: Leveraging Log Rank Information for Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2306.05540/) (2023)
- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [Intrinsic Dimension Estimation for Robust Detection of AI-Generated Texts](https://mlsanity.com/paper/2306.04723/) (2023)
- [Language GANs Falling Short](https://mlsanity.com/paper/1811.02549/) (2018)
- [Trading Off Diversity and Quality in Natural Language Generation](https://mlsanity.com/paper/2004.10450/) (2020)
- [Detecting Subtle Differences between Human and Model Languages Using Spectrum of Relative Likelihood](https://mlsanity.com/paper/2406.19874/) (2024)
- [Detecting Machine-Generated Texts by Multi-Population Aware Optimization for Maximum Mean Discrepancy](https://mlsanity.com/paper/2402.16041/) (2024)
- [Detecting Fake News Using Machine Learning : A Systematic Literature Review](https://mlsanity.com/paper/2102.04458/) (2021)
- [On the Efficacy of Sampling Adapters](https://mlsanity.com/paper/2307.03749/) (2023)
- [Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text](https://mlsanity.com/paper/2405.12689/) (2024)
- [FACE: Evaluating Natural Language Generation with Fourier Analysis of Cross-Entropy](https://mlsanity.com/paper/2305.10307/) (2023)
- [On Evaluating The Performance of Watermarked Machine-Generated Texts Under Adversarial Attacks](https://mlsanity.com/paper/2407.04794/) (2024)

## Built on by (most cited first, 3 of 3)

- [Local Normalization Distortion and the Thermodynamic Formalism of Decoding Strategies for Large Language Models](https://mlsanity.com/paper/2503.21929/) (2025)
- [DMAP: A Distribution Map for Text](https://mlsanity.com/paper/2602.11871/) (2026)
- [Log-Likelihood, Simpson's Paradox, and the Detection of Machine-Generated Text](https://mlsanity.com/paper/2605.06294/) (2026)

## Abstract (arXiv)

Existing methods for the zero-shot detection of machine-generated text are dominated by three statistical quantities: log-likelihood, log-rank, and entropy. As language models mimic the distribution of human text ever closer, this will limit our ability to build effective detection algorithms. To combat this, we introduce a method for detecting machine-generated text that is entirely agnostic of the generating language model. This is achieved by targeting a defect in the way that decoding strategies, such as temperature or top-k sampling, normalize conditional probability measures. This method can be rigorously theoretically justified, is easily explainable, and is conceptually distinct from existing methods for detecting machine-generated text. We evaluate our detector in the white and black box settings across various language models, datasets, and passage lengths. We also study the effect of paraphrasing attacks on our detector and the extent to which it is biased against non-native speakers. In each of these settings, the performance of our test is at least comparable to that of other state-of-the-art text detectors, and in some cases, we strongly outperform these baselines.