# Deciphering Textual Authenticity: A Generalized Strategy through the Lens of Large Language Semantics for Detecting Human vs. Machine-Generated Text arXiv: https://arxiv.org/abs/2401.09407 ยท HTML: https://mlsanity.com/paper/2401.09407/ ## Key facts - Cited by 5 papers in the map (data through 14 Oct 2026). - Builds on 32 of its 58 references that are in the map. - First posted on arXiv 17 Jan 2024. ## Builds on (most cited first, 30 of 32) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019) - [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019) - [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023) - [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019) - [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015) - [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018) - [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023) - [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022) - [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019) - [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019) - [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019) - [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023) - [Text and Code Embeddings by Contrastive Pre-Training](https://mlsanity.com/paper/2201.10005/) (2022) - [Black-box Generation of Adversarial Text Sequences to Evade Deep Learning Classifiers](https://mlsanity.com/paper/1801.04354/) (2018) - [Fast-DetectGPT: Efficient Zero-Shot Detection of Machine-Generated Text via Conditional Probability Curvature](https://mlsanity.com/paper/2310.05130/) (2023) - [PromptRobust: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts](https://mlsanity.com/paper/2306.04528/) (2023) - [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023) - [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023) - [Distilling Reasoning Capabilities into Smaller Language Models](https://mlsanity.com/paper/2212.00193/) (2022) - [M4: Multi-generator, Multi-domain, and Multi-lingual Black-Box Machine-Generated Text Detection](https://mlsanity.com/paper/2305.14902/) (2023) - [TweepFake: about Detecting Deepfake Tweets](https://mlsanity.com/paper/2008.00036/) (2020) - [MGTBench: Benchmarking Machine-Generated Text Detection](https://mlsanity.com/paper/2303.14822/) (2023) - [Do Massively Pretrained Language Models Make Better Storytellers?](https://mlsanity.com/paper/1909.10705/) (2019) - [Machine Generated Text: A Comprehensive Survey of Threat Models and Detection Methods](https://mlsanity.com/paper/2210.07321/) (2022) - [Artificial Text Detection via Examining the Topology of Attention Maps](https://mlsanity.com/paper/2109.04825/) (2021) - [GPT-Sentinel: Distinguishing Human and ChatGPT Generated Content](https://mlsanity.com/paper/2305.07969/) (2023) - [Deepfake Text Detection: Limitations and Opportunities](https://mlsanity.com/paper/2210.09421/) (2022) - [The Looming Threat of Fake and LLM-generated LinkedIn Profiles: Challenges and Opportunities for Detection and Prevention](https://mlsanity.com/paper/2307.11864/) (2023) - [CIFAR-10: KNN-based Ensemble of Classifiers](https://mlsanity.com/paper/1611.04905/) (2016) ## Built on by (most cited first, 5 of 5) - [Lateral Phishing With Large Language Models: A Large Organization Comparative Study](https://mlsanity.com/paper/2401.09727/) (2024) - [ESPERANTO: Evaluating Synthesized Phrases to Enhance Robustness in AI Detection for Text Origination](https://mlsanity.com/paper/2409.14285/) (2024) - [A Lightweight Approach to Detection of AI-Generated Texts Using Stylometric Features](https://mlsanity.com/paper/2511.21744/) (2025) - [Synthetic Artifact Auditing: Tracing LLM-Generated Synthetic Data Usage in Downstream Applications](https://mlsanity.com/paper/2502.00808/) (2025) - [AI-generated Text Detection: A Multifaceted Approach to Binary and Multiclass Classification](https://mlsanity.com/paper/2505.11550/) (2025) ## Abstract (arXiv) With the recent proliferation of Large Language Models (LLMs), there has been an increasing demand for tools to detect machine-generated text. The effective detection of machine-generated text face two pertinent problems: First, they are severely limited in generalizing against real-world scenarios, where machine-generated text is produced by a variety of generators, including but not limited to GPT-4 and Dolly, and spans diverse domains, ranging from academic manuscripts to social media posts. Second, existing detection methodologies treat texts produced by LLMs through a restrictive binary classification lens, neglecting the nuanced diversity of artifacts generated by different LLMs. In this work, we undertake a systematic study on the detection of machine-generated text in real-world scenarios. We first study the effectiveness of state-of-the-art approaches and find that they are severely limited against text produced by diverse generators and domains in the real world. Furthermore, t-SNE visualizations of the embeddings from a pretrained LLM's encoder show that they cannot reliably distinguish between human and machine-generated text. Based on our findings, we introduce a novel system, T5LLMCipher, for detecting machine-generated text using a pretrained T5 encoder combined with LLM embedding sub-clustering to address the text produced by diverse generators and domains in the real world. We evaluate our approach across 9 machine-generated text systems and 9 domains and find that our approach provides state-of-the-art generalization ability, with an average increase in F1 score on machine-generated text of 19.6% on unseen generators and domains compared to the top performing existing approaches and correctly attributes the generator of text with an accuracy of 93.6%.