# Human heuristics for AI-generated language are flawed

arXiv: https://arxiv.org/abs/2206.07271 · HTML: https://mlsanity.com/paper/2206.07271/

## Key facts

- Cited by 105 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 54 references that are in the map.
- First posted on arXiv 15 Jun 2022.

## Builds on (most cited first, 11 of 11)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [Unifying Human and Statistical Evaluation for Natural Language Generation](https://mlsanity.com/paper/1904.02792/) (2019)
- [The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation](https://mlsanity.com/paper/2109.06835/) (2021)
- [Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Text](https://mlsanity.com/paper/2107.01294/) (2021)

## Built on by (most cited first, 30 of 105)

- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [Generative AI](https://mlsanity.com/paper/2309.07930/) (2023)
- [Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Tasks](https://mlsanity.com/paper/2306.07899/) (2023)
- [Language Model Behavior: A Comprehensive Survey](https://mlsanity.com/paper/2303.11504/) (2023)
- [A Design Space for Intelligent and Interactive Writing Assistants](https://mlsanity.com/paper/2403.14117/) (2024)
- [Large Language Model Safety: A Holistic Survey](https://mlsanity.com/paper/2412.17686/) (2024)
- [Large Language Models Assume People are More Rational than We Really are](https://mlsanity.com/paper/2406.17055/) (2024)
- [The Persuasive Power of Large Language Models](https://mlsanity.com/paper/2312.15523/) (2023)
- [Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?](https://mlsanity.com/paper/2308.01284/) (2023)
- [Anatomy of an AI-powered malicious social botnet](https://mlsanity.com/paper/2307.16336/) (2023)
- [Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languages](https://mlsanity.com/paper/2310.17526/) (2023)
- [Embarrassingly Simple Text Watermarks](https://mlsanity.com/paper/2310.08920/) (2023)
- [How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experiment](https://mlsanity.com/paper/2401.13481/) (2024)
- [AI vs. Human -- Differentiation Analysis of Scientific Content Generation](https://mlsanity.com/paper/2301.10416/) (2023)
- [Ten Words Only Still Help: Improving Black-Box AI-Generated Text Detection via Proxy-Guided Efficient Re-Sampling](https://mlsanity.com/paper/2402.09199/) (2024)
- [Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)](https://mlsanity.com/paper/2505.21091/) (2025)
- [Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Models](https://mlsanity.com/paper/2412.17128/) (2024)
- [Characteristics and prevalence of fake social media profiles with AI-generated faces](https://mlsanity.com/paper/2401.02627/) (2024)
- [Large language models can consistently generate high-quality content for election disinformation operations](https://mlsanity.com/paper/2408.06731/) (2024)
- ["It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Models](https://mlsanity.com/paper/2411.13032/) (2024)
- [Characterizing Photorealism and Artifacts in Diffusion Model-Generated Images](https://mlsanity.com/paper/2502.11989/) (2025)
- [Labeling Messages as AI-Generated Does Not Reduce Their Persuasive Effects](https://mlsanity.com/paper/2504.09865/) (2025)
- [The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Making](https://mlsanity.com/paper/2410.07304/) (2024)
- [Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Text](https://mlsanity.com/paper/2407.09364/) (2024)
- [MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checking](https://mlsanity.com/paper/2407.13089/) (2024)
- [Benchmarking the Detection of LLMs-Generated Modern Chinese Poetry](https://mlsanity.com/paper/2509.01620/) (2025)
- [Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communication](https://mlsanity.com/paper/2302.13382/) (2023)
- [A Linguistic Comparison between Human and ChatGPT-Generated Conversations](https://mlsanity.com/paper/2401.16587/) (2024)
- [Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoption](https://mlsanity.com/paper/2409.03843/) (2024)

## Abstract (arXiv)

Human communication is increasingly intermixed with language generated by AI. Across chat, email, and social media, AI systems suggest words, complete sentences, or produce entire conversations. AI-generated language is often not identified as such but presented as language written by humans, raising concerns about novel forms of deception and manipulation. Here, we study how humans discern whether verbal self-presentations, one of the most personal and consequential forms of language, were generated by AI. In six experiments, participants (N = 4,600) were unable to detect self-presentations generated by state-of-the-art AI language models in professional, hospitality, and dating contexts. A computational analysis of language features shows that human judgments of AI-generated language are hindered by intuitive but flawed heuristics such as associating first-person pronouns, use of contractions, or family topics with human-written language. We experimentally demonstrate that these heuristics make human judgment of AI-generated language predictable and manipulable, allowing AI systems to produce text perceived as "more human than human." We discuss solutions, such as AI accents, to reduce the deceptive potential of language generated by AI, limiting the subversion of human intuition.