mlsanityData to 7 Oct 2026

Human heuristics for AI-generated language are flawed

arXiv 2206.07271 ↗First posted 15 Jun 2022cs.CL cs.AI cs.CYcited by 104

Not on the map. Why?

Abstract (arXiv)

Human communication is increasingly intermixed with language generated by AI. Across chat, email, and social media, AI systems suggest words, complete sentences, or produce entire conversations. AI-generated language is often not identified as such but presented as language written by humans, raising concerns about novel forms of deception and manipulation. Here, we study how humans discern whether verbal self-presentations, one of the most personal and consequential forms of language, were generated by AI. In six experiments, participants (N = 4,600) were unable to detect self-presentations generated by state-of-the-art AI language models in professional, hospitality, and dating contexts. A computational analysis of language features shows that human judgments of AI-generated language are hindered by intuitive but flawed heuristics such as associating first-person pronouns, use of contractions, or family topics with human-written language. We experimentally demonstrate that these heuristics make human judgment of AI-generated language predictable and manipulable, allowing AI systems to produce text perceived as "more human than human." We discuss solutions, such as AI accents, to reduce the deceptive potential of language generated by AI, limiting the subversion of human intuition.

Key facts. Cited by 104 papers in the map (data through 7 Oct 2026). Builds on 11 of its 54 references that are in the map. First posted on arXiv 15 Jun 2022.

Builds on

11 of 54 references.

References · 11

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2019Defending Against Neural Fake Newscited by 553
  6. 2019GLTR: Statistical Detection and Visualization of Generated Textcited by 490
  7. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  8. 2019Automatic Detection of Generated Text is Easiest when Humans are Fooledcited by 260
  9. 2019Unifying Human and Statistical Evaluation for Natural Language Generationcited by 127
  10. 2021The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generationcited by 96
  11. 2021Is GPT-3 Text Indistinguishable from Human Text? Scarecrow: A Framework for Scrutinizing Machine Textcited by 89

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 17; 2024: 30; 2025: 27; 2026: 30.

104 papers.

Sort:

By area:

  1. 2023Co-Writing with Opinionated Language Models Affects Users' Viewscited by 145
  2. 2023AI Transparency in the Age of LLMs: A Human-Centered Research Roadmapcited by 119
  3. 2023Generative AIcited by 119
  4. 2023Artificial Artificial Artificial Intelligence: Crowd Workers Widely Use Large Language Models for Text Production Taskscited by 102
  5. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  6. 2024A Design Space for Intelligent and Interactive Writing Assistantscited by 76
  7. 2024Large Language Models Assume People are More Rational than We Really arecited by 49
  8. 2023The Persuasive Power of Large Language Modelscited by 46
  9. 2023Fighting Fire with Fire: Can ChatGPT Detect AI-generated Text?cited by 39
  10. 2023Can large language models replace humans in the systematic review process? Evaluating GPT-4's efficacy in screening and extracting data from peer-reviewed and grey literature in multiple languagescited by 36
  11. 2023Anatomy of an AI-powered malicious social botnetcited by 35
  12. 2023Embarrassingly Simple Text Watermarkscited by 23
38 more
  1. 2024How AI Ideas Affect the Creativity, Diversity, and Evolution of Human Ideas: Evidence From a Large, Dynamic Experimentcited by 22
  2. 2023AI vs. Human -- Differentiation Analysis of Scientific Content Generationcited by 21
  3. 2024Ten Words Only Still Help: Improving Black-Box AI-Generated Text Detection via Proxy-Guided Efficient Re-Samplingcited by 20
  4. 2025Position is Power: System Prompts as a Mechanism of Bias in Large Language Models (LLMs)cited by 19
  5. 2024Characteristics and prevalence of fake social media profiles with AI-generated facescited by 14
  6. 2024"It was 80% me, 20% AI": Seeking Authenticity in Co-Writing with Large Language Modelscited by 14
  7. 2024Lies, Damned Lies, and Distributional Language Statistics: Persuasion and Deception with Large Language Modelscited by 14
  8. 2024Large language models can consistently generate high-quality content for election disinformation operationscited by 13
  9. 2025Characterizing Photorealism and Artifacts in Diffusion Model-Generated Imagescited by 13
  10. 2025Labeling Messages as AI-Generated Does Not Reduce Their Persuasive Effectscited by 13
  11. 2024The Moral Turing Test: Evaluating Human-LLM Alignment in Moral Decision-Makingcited by 12
  12. 2024Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Textcited by 11
  13. 2024MetaSumPerceiver: Multimodal Multi-Document Evidence Summarization for Fact-Checkingcited by 10
  14. 2025Benchmarking the Detection of LLMs-Generated Modern Chinese Poetrycited by 10
  15. 2023Comparing Sentence-Level Suggestions to Message-Level Suggestions in AI-Mediated Communicationcited by 9
  16. 2024A Linguistic Comparison between Human and ChatGPT-Generated Conversationscited by 9
  17. 2024Persona Setting Pitfall: Persistent Outgroup Biases in Large Language Models Arising from Social Identity Adoptioncited by 9
  18. 2024Examining the Prevalence and Dynamics of AI-Generated Media in Art Subredditscited by 8
  19. 2025OpenTuringBench: An Open-Model-based Benchmark and Framework for Machine-Generated Text Detection and Attributioncited by 8
  20. 2024People over trust AI-generated medical responses and view them to be as valid as doctors, despite low accuracycited by 7
  21. 2025Authorship Attribution in Multilingual Machine-Generated Textscited by 7
  22. 2024Exploring Bias and Prediction Metrics to Characterise the Fairness of Machine Learning for Equity-Centered Public Health Decision-Making: A Narrative Reviewcited by 6
  23. 2025Poor Alignment and Steerability of Large Language Models: Evidence from College Admission Essayscited by 6
  24. 2025Linguistic and Embedding-Based Profiling of Texts generated by Humans and Large Language Modelscited by 6
  25. 2026The Impact of AI-Generated Text on the Internetcited by 6
  26. 2024I Am Not Them: Fluid Identities and Persistent Out-group Bias in Large Language Modelscited by 5
  27. 2024MAiDE-up: Multilingual Deception Detection of GPT-generated Hotel Reviewscited by 5
  28. 2024LLM Detectors Still Fall Short of Real World: Case of LLM-Generated Short News-Like Postscited by 4
  29. 2026Reactive Writers: How Co-Writing with AI Changes How We Engage with Ideascited by 4
  30. 2026Social Simulacra in the Wild: AI Agent Communities on Moltbookcited by 4
  31. 2026"I understand your perspective": LLM Persuasion through the Lens of Communicative Action Theorycited by 4
  32. 2024Cross-cultural Inspiration Detection and Analysis in Real and LLM-generated Social Media Datacited by 3
  33. 2024From Complexity to Clarity: How AI Enhances Perceptions of Scientists and the Public's Understanding of Sciencecited by 3
  34. 2024Towards Realistic Synthetic User-Generated Content: A Scaffolding Approach to Generating Online Discussionscited by 3
  35. 2024Cognitive phantoms in LLMs through the lens of latent variablescited by 3
  36. 2024Fool Me, Fool Me: User Attitudes Toward LLM Falsehoodscited by 3
  37. 2025Collaborative Evaluation of Deepfake Text with Deliberation-Enhancing Dialogue Systemscited by 3
  38. 2025Web-Browsing LLMs Can Access Social Media Profiles and Infer User Demographicscited by 3

50 most cited of 104 All 104 (JSON)