# Large Language Models can be Guided to Evade AI-Generated Text Detection

arXiv: https://arxiv.org/abs/2305.10847 · HTML: https://mlsanity.com/paper/2305.10847/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 51 references that are in the map.
- First posted on arXiv 18 May 2023.

## Builds on (most cited first, 30 of 32)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [Character-level Convolutional Networks for Text Classification](https://mlsanity.com/paper/1509.01626/) (2015)
- [What Makes Good In-Context Examples for GPT-3?](https://mlsanity.com/paper/2101.06804/) (2021)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [Learning To Retrieve Prompts for In-Context Learning](https://mlsanity.com/paper/2112.08633/) (2021)
- [MTEB: Massive Text Embedding Benchmark](https://mlsanity.com/paper/2210.07316/) (2022)
- [ELI5: Long Form Question Answering](https://mlsanity.com/paper/1907.09190/) (2019)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [How Close is ChatGPT to Human Experts? Comparison Corpus, Evaluation, and Detection](https://mlsanity.com/paper/2301.07597/) (2023)
- [Sentence-T5: Scalable Sentence Encoders from Pre-trained Text-to-Text Models](https://mlsanity.com/paper/2108.08877/) (2021)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Demystifying Prompts in Language Models via Perplexity Estimation](https://mlsanity.com/paper/2212.04037/) (2022)
- [Adversarial Watermarking Transformer: Towards Tracing Text Provenance with Data Hiding](https://mlsanity.com/paper/2009.03015/) (2020)
- [Self-Generated In-Context Learning: Leveraging Auto-regressive Language Models as a Demonstration Generator](https://mlsanity.com/paper/2206.08082/) (2022)
- [Generating Sentiment-Preserving Fake Online Reviews Using Neural Language Models and Their Human- and Machine-based Detection](https://mlsanity.com/paper/1907.09177/) (2019)
- [Smaller Language Models are Better Black-box Machine-Generated Text Detectors](https://mlsanity.com/paper/2305.09859/) (2023)

## Built on by (most cited first, 30 of 44)

- [RAID: A Shared Benchmark for Robust Evaluation of Machine-Generated Text Detectors](https://mlsanity.com/paper/2405.07940/) (2024)
- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [A Survey on Detection of LLMs-Generated Content](https://mlsanity.com/paper/2310.15654/) (2023)
- [People who frequently use ChatGPT for writing tasks are accurate and robust detectors of AI-generated text](https://mlsanity.com/paper/2501.15654/) (2025)
- [DetectRL: Benchmarking LLM-Generated Text Detection in Real-World Scenarios](https://mlsanity.com/paper/2410.23746/) (2024)
- [Securing Large Language Models: Threats, Vulnerabilities and Responsible Practices](https://mlsanity.com/paper/2403.12503/) (2024)
- [Bypassing LLM Watermarks with Color-Aware Substitutions](https://mlsanity.com/paper/2403.14719/) (2024)
- [What Does the Bot Say? Opportunities and Risks of Large Language Models in Social Media Bot Detection](https://mlsanity.com/paper/2402.00371/) (2024)
- [A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents](https://mlsanity.com/paper/2402.10196/) (2024)
- [Authorship Obfuscation in Multilingual Machine-Generated Text Detection](https://mlsanity.com/paper/2401.07867/) (2024)
- [User Modeling in the Era of Large Language Models: Current Research and Future Directions](https://mlsanity.com/paper/2312.11518/) (2023)
- [Safe Delta: Consistently Preserving Safety when Fine-Tuning LLMs on Diverse Datasets](https://mlsanity.com/paper/2505.12038/) (2025)
- [Large Language Models are Fixated by Red Herrings: Exploring Creative Problem Solving and Einstellung Effect using the Only Connect Wall Dataset](https://mlsanity.com/paper/2306.11167/) (2023)
- [Spotting AI's Touch: Identifying LLM-Paraphrased Spans in Text](https://mlsanity.com/paper/2405.12689/) (2024)
- [A Survey of Attacks on Large Language Models](https://mlsanity.com/paper/2505.12567/) (2025)
- [Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Text](https://mlsanity.com/paper/2403.05750/) (2024)
- [SilverSpeak: Evading AI-Generated Text Detectors using Homoglyphs](https://mlsanity.com/paper/2406.11239/) (2024)
- [Hidding the Ghostwriters: An Adversarial Evaluation of AI-Generated Student Essay Detection](https://mlsanity.com/paper/2402.00412/) (2024)
- [Your Language Model Can Secretly Write Like Humans: Contrastive Paraphrase Attacks on LLM-Generated Text Detectors](https://mlsanity.com/paper/2505.15337/) (2025)
- [Frankentext: Stitching random text fragments into long-form narratives](https://mlsanity.com/paper/2505.18128/) (2025)
- [The Impact of Prompts on Zero-Shot Detection of AI-Generated Text](https://mlsanity.com/paper/2403.20127/) (2024)
- [ESPERANTO: Evaluating Synthesized Phrases to Enhance Robustness in AI Detection for Text Origination](https://mlsanity.com/paper/2409.14285/) (2024)
- [Humanizing the Machine: Proxy Attacks to Mislead LLM Detectors](https://mlsanity.com/paper/2410.19230/) (2024)
- [Survey on Plagiarism Detection in Large Language Models: The Impact of ChatGPT and Gemini on Academic Integrity](https://mlsanity.com/paper/2407.13105/) (2024)
- [FDLLM: A Dedicated Detector for Black-Box LLMs Fingerprinting](https://mlsanity.com/paper/2501.16029/) (2025)
- [How Sampling Affects the Detectability of Machine-written texts: A Comprehensive Study](https://mlsanity.com/paper/2510.13681/) (2025)
- [PADBen: A Comprehensive Benchmark for Evaluating AI Text Detectors Against Paraphrase Attacks](https://mlsanity.com/paper/2511.00416/) (2025)
- [AHD Agent: Agentic Reinforcement Learning for Automatic Heuristic Design](https://mlsanity.com/paper/2605.08756/) (2026)
- [Navigating the Shadows: Unveiling Effective Disturbances for Modern AI Content Detectors](https://mlsanity.com/paper/2406.08922/) (2024)
- [Open Your Eyes: Vision Enhances Message Passing Neural Networks in Link Prediction](https://mlsanity.com/paper/2505.08266/) (2025)

## Abstract (arXiv)

Large language models (LLMs) have shown remarkable performance in various tasks and have been extensively utilized by the public. However, the increasing concerns regarding the misuse of LLMs, such as plagiarism and spamming, have led to the development of multiple detectors, including fine-tuned classifiers and statistical methods. In this study, we equip LLMs with prompts, rather than relying on an external paraphraser, to evaluate the vulnerability of these detectors. We propose a novel Substitution-based In-Context example Optimization method (SICO) to automatically construct prompts for evading the detectors. SICO is cost-efficient as it requires only 40 human-written examples and a limited number of LLM inferences to generate a prompt. Moreover, once a task-specific prompt has been constructed, it can be universally used against a wide range of detectors. Extensive experiments across three real-world tasks demonstrate that SICO significantly outperforms the paraphraser baselines and enables GPT-3.5 to successfully evade six detectors, decreasing their AUC by 0.5 on average. Furthermore, a comprehensive human evaluation show that the SICO-generated text achieves human-level readability and task completion rates, while preserving high imperceptibility. Finally, we propose an ensemble approach to enhance the robustness of detectors against SICO attack. The code is publicly available at https://github.com/ColinLu50/Evade-GPT-Detector.