# LLMDet: A Third Party Large Language Models Generated Text Detection Tool

arXiv: https://arxiv.org/abs/2305.15004 · HTML: https://mlsanity.com/paper/2305.15004/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 36 references that are in the map.
- First posted on arXiv 24 May 2023.

## Builds on (most cited first, 25 of 25)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [BLOOM: A 176B-Parameter Open-Access Multilingual Language Model](https://mlsanity.com/paper/2211.05100/) (2022)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [GLM: General Language Model Pretraining with Autoregressive Blank Infilling](https://mlsanity.com/paper/2103.10360/) (2021)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Weight Poisoning Attacks on Pre-trained Models](https://mlsanity.com/paper/2004.06660/) (2020)
- [Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding](https://mlsanity.com/paper/2009.03015/) (2020)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [TweepFake: about Detecting Deepfake Tweets](https://mlsanity.com/paper/2008.00036/) (2020)
- [Real or Fake? Learning to Discriminate Machine from Human Generated Text](https://mlsanity.com/paper/1906.03351/) (2019)
- [On the Possibilities of AI-Generated Text Detection](https://mlsanity.com/paper/2304.04736/) (2023)
- [Text Matching as Image Recognition](https://mlsanity.com/paper/1602.06359/) (2016)
- [Will Large-scale Generative Models Corrupt Future Datasets?](https://mlsanity.com/paper/2211.08095/) (2022)
- [Frustratingly Easy Edit-based Linguistic Steganography with a Masked Language Model](https://mlsanity.com/paper/2104.09833/) (2021)
- [Perplexed by Quality: A Perplexity-based Method for Adult and Harmful Content Detection in Multilingual Heterogeneous Web Data](https://mlsanity.com/paper/2212.10440/) (2022)
- [OPT: Omni-Perception Pre-Trainer for Cross-Modal Understanding and Generation](https://mlsanity.com/paper/2107.00249/) (2021)

## Built on by (most cited first, 30 of 45)

- [RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://mlsanity.com/paper/2405.07940/) (2024)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [WaterMax: breaking the LLM watermark detectability-robustness-quality trade-off](https://mlsanity.com/paper/2403.04808/) (2024)
- [Token-Specific Watermarking with Enhanced Detectability and Semantic Coherence for Large Language Models](https://mlsanity.com/paper/2402.18059/) (2024)
- [Between Lines of Code: Unraveling the Distinct Patterns of Machine and Human Programmers](https://mlsanity.com/paper/2401.06461/) (2024)
- [AI-generated text boundary detection with RoFT](https://mlsanity.com/paper/2311.08349/) (2023)
- [A Survey of AI-generated Text Forensic Systems: Detection, Attribution, and Characterization](https://mlsanity.com/paper/2403.01152/) (2024)
- [Almost AI, Almost Human: The Challenge of Detecting AI-Polished Writing](https://mlsanity.com/paper/2502.15666/) (2025)
- [Attacks on Third-Party APIs of Large Language Models](https://mlsanity.com/paper/2404.16891/) (2024)
- [Detecting Machine-Generated Texts: Not Just "AI vs Humans" and Explainability is Complicated](https://mlsanity.com/paper/2406.18259/) (2024)
- [Near to Mid-term Risks and Opportunities of Open-Source Generative AI](https://mlsanity.com/paper/2404.17047/) (2024)
- [Risks and Opportunities of Open-Source Generative AI](https://mlsanity.com/paper/2405.08597/) (2024)
- [Beyond Next Token Probabilities: Learnable, Fast Detection of Hallucinations and Data Contamination on LLM Output Distributions](https://mlsanity.com/paper/2503.14043/) (2025)
- [Does DetectGPT Fully Utilize Perturbation? Bridging Selective Perturbation to Fine-tuned Contrastive Learning Detector would be Better](https://mlsanity.com/paper/2402.00263/) (2024)
- ['Quis custodiet ipsos custodes?' Who will watch the watchmen? On Detecting AI-generated peer-reviews](https://mlsanity.com/paper/2410.09770/) (2024)
- [SCAR: Data Selection via Style Consistency-Aware Response Ranking for Efficient Instruction-Tuning of Large Language Models](https://mlsanity.com/paper/2406.10882/) (2024)
- [ESPERANTO: Evaluating Synthesized Phrases to Enhance Robustness in AI Detection for Text Origination](https://mlsanity.com/paper/2409.14285/) (2024)
- [Large Language Model Sourcing: A Survey](https://mlsanity.com/paper/2510.10161/) (2025)
- [Detecting LLM-assisted writing in scientific communication: Are we there yet?](https://mlsanity.com/paper/2401.16807/) (2024)
- [Synthetic Lyrics Detection Across Languages and Genres](https://mlsanity.com/paper/2406.15231/) (2024)
- [GigaCheck: Detecting LLM-generated Content via Object-Centric Span Localization](https://mlsanity.com/paper/2410.23728/) (2024)
- [Not all tokens are created equal: Perplexity Attention Weighted Networks for AI generated text detection](https://mlsanity.com/paper/2501.03940/) (2025)
- [Who Writes What: Unveiling the Impact of Author Roles on AI-generated Text Detection](https://mlsanity.com/paper/2502.12611/) (2025)
- [Recent Advances in Transformer and Large Language Models for UAV Applications](https://mlsanity.com/paper/2508.11834/) (2025)
- [CoCoNUTS: Concentrating on Content while Neglecting Uninformative Textual Styles for AI-Generated Peer Review Detection](https://mlsanity.com/paper/2509.04460/) (2025)
- [On the Detectability of LLM-Generated Text: What Exactly Is LLM-Generated Text?](https://mlsanity.com/paper/2510.20810/) (2025)
- [Kill two birds with one stone: generalized and robust AI-generated text detection via dynamic perturbations](https://mlsanity.com/paper/2504.21019/) (2025)
- [MSPL: Multi-Step Pseudo-Labeling for Open-Vocabulary Object Detection](https://mlsanity.com/paper/2510.14792/) (2025)
- [Spotlights and Blindspots: Evaluating Machine-Generated Text Detection](https://mlsanity.com/paper/2604.16607/) (2026)
- [Beyond Turing: A Comparative Analysis of Approaches for Detecting Machine-Generated Text](https://mlsanity.com/paper/2311.12373/) (2023)

## Abstract (arXiv)

Generated texts from large language models (LLMs) are remarkably close to high-quality human-authored text, raising concerns about their potential misuse in spreading false information and academic misconduct. Consequently, there is an urgent need for a highly practical detection tool capable of accurately identifying the source of a given text. However, existing detection tools typically rely on access to LLMs and can only differentiate between machine-generated and human-authored text, failing to meet the requirements of fine-grained tracing, intermediary judgment, and rapid detection. Therefore, we propose LLMDet, a model-specific, secure, efficient, and extendable detection tool, that can source text from specific LLMs, such as GPT-2, OPT, LLaMA, and others. In LLMDet, we record the next-token probabilities of salient n-grams as features to calculate proxy perplexity for each LLM. By jointly analyzing the proxy perplexities of LLMs, we can determine the source of the generated text. Experimental results show that LLMDet yields impressive detection performance while ensuring speed and security, achieving 98.54% precision and x5.0 faster for recognizing human-authored text. Additionally, LLMDet can effortlessly extend its detection capabilities to a new open-source model. We will provide an open-source tool at https://github.com/TrustedLLM/LLMDet.