# Reverse Engineering Configurations of Neural Text Generation Models

arXiv: https://arxiv.org/abs/2004.06201 · HTML: https://mlsanity.com/paper/2004.06201/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Apr 2020.

## Built on by (most cited first, 15 of 15)

- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Unbiased Watermark for Large Language Models](https://mlsanity.com/paper/2310.10669/) (2023)
- [Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods](https://mlsanity.com/paper/2210.07321/) (2022)
- [A Resilient and Accessible Distribution-Preserving Watermark for Large Language Models](https://mlsanity.com/paper/2310.07710/) (2023)
- [CHEAT: A Large-scale Dataset for Detecting ChatGPT-writtEn AbsTracts](https://mlsanity.com/paper/2304.12008/) (2023)
- [CodeIP: A Grammar-Guided Multi-Bit Watermark for Large Language Models of Code](https://mlsanity.com/paper/2404.15639/) (2024)
- [Exploring and Mitigating Adversarial Manipulation of Voting-Based Leaderboards](https://mlsanity.com/paper/2501.07493/) (2025)
- [Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text](https://mlsanity.com/paper/2309.07689/) (2023)
- [Distortion-free Watermarks are not Truly Distortion-free under Watermark Key Collisions](https://mlsanity.com/paper/2406.02603/) (2024)
- [Reverse-Engineering Decoding Strategies Given Blackbox Access to a Language Generation System](https://mlsanity.com/paper/2309.04858/) (2023)
- [How well can machine-generated texts be identified and can language models be trained to avoid identification?](https://mlsanity.com/paper/2310.16992/) (2023)
- [Detection Avoidance Techniques for Large Language Models](https://mlsanity.com/paper/2503.07595/) (2025)
- [DSIPA: Detecting LLM-Generated Texts via Sentiment-Invariant Patterns Divergence Analysis](https://mlsanity.com/paper/2604.26328/) (2026)
- [Improving Detection of Watermarked Language Models](https://mlsanity.com/paper/2508.13131/) (2025)
- [InterPol: De-anonymizing LM Arena via Interpolated Preference Learning](https://mlsanity.com/paper/2603.15220/) (2026)

## Abstract (arXiv)

This paper seeks to develop a deeper understanding of the fundamental properties of neural text generations models. The study of artifacts that emerge in machine generated text as a result of modeling choices is a nascent research area. Previously, the extent and degree to which these artifacts surface in generated text has not been well studied. In the spirit of better understanding generative text models and their artifacts, we propose the new task of distinguishing which of several variants of a given model generated a piece of text, and we conduct an extensive suite of diagnostic tests to observe whether modeling choices (e.g., sampling methods, top-k probabilities, model architectures, etc.) leave detectable artifacts in the text they generate. Our key finding, which is backed by a rigorous set of experiments, is that such artifacts are present and that different modeling choices can be inferred by observing the generated text alone. This suggests that neural text generators may be more sensitive to various modeling choices than previously thought.