# Can Humans Tell? A Dual-Axis Study of Human Perception of LLM-Generated News

arXiv: https://arxiv.org/abs/2604.03755 · HTML: https://mlsanity.com/paper/2604.03755/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 4 Apr 2026.

## Builds on (most cited first, 10 of 10)

- [Defending Against Neural Fake News](https://mlsanity.com/paper/1905.12616/) (2019)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Human heuristics for AI-generated language are flawed](https://mlsanity.com/paper/2206.07271/) (2022)
- [The science of fake news](https://mlsanity.com/paper/2307.07903/) (2023)
- [GenAI Against Humanity: Nefarious Applications of Generative Artificial Intelligence and Large Language Models](https://mlsanity.com/paper/2310.00737/) (2023)
- [Blessing or curse? A survey on the Impact of Generative AI on Fake News](https://mlsanity.com/paper/2404.03021/) (2024)
- [Industrialized Deception: The Collateral Effects of LLM-Generated Misinformation on Digital Ecosystems](https://mlsanity.com/paper/2601.21963/) (2026)
- [Eroding the Truth-Default: A Causal Analysis of Human Susceptibility to Foundation Model Hallucinations and Disinformation in the Wild](https://mlsanity.com/paper/2601.22871/) (2026)
- [Origin Lens: A Privacy-First Mobile Framework for Cryptographic Image Provenance and AI Detection](https://mlsanity.com/paper/2602.03423/) (2026)
- [CRED-1: An Open Multi-Signal Domain Credibility Dataset for Automated Pre-Bunking of Online Misinformation](https://mlsanity.com/paper/2604.20856/) (2026)

## Built on by (most cited first, 1 of 1)

- [Interrupting the Chain: Human Perception of AI-Generated Disinformation Through a Kill Chain Lens](https://mlsanity.com/paper/2608.21389/) (2026)

## Abstract (arXiv)

Can humans tell whether a news article was written by a person or a large language model (LLM)? We investigate this question using JudgeGPT, a study platform that independently measures source attribution (human vs. machine) and authenticity judgment (legitimate vs. fake) on continuous scales. From 2,318 judgments collected from 1,054 participants across content generated by six LLMs, we report five findings: (1) participants cannot reliably distinguish machine-generated from human-written text (p > .05, Welch's t-test); (2) this inability holds across all tested models, including open-weight models with as few as 7B parameters; (3) self-reported domain expertise predicts judgment accuracy (r = .35, p < .001) whereas political orientation does not (r = -.10, n.s.); (4) clustering reveals distinct response strategies ("Skeptics" vs. "Believers"); and (5) accuracy degrades after approximately 30 sequential evaluations due to cognitive fatigue. The answer, in short, is no: humans cannot reliably tell. These results indicate that user-side detection is not a viable defense and motivate system-level countermeasures such as cryptographic content provenance.