# Humans can learn to detect AI-generated texts, or at least learn when they can't

arXiv: https://arxiv.org/abs/2505.01877 · HTML: https://mlsanity.com/paper/2505.01877/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 24 references that are in the map.
- First posted on arXiv 3 May 2025.

## Builds on (most cited first, 4 of 4)

- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Human heuristics for AI-generated language are flawed](https://mlsanity.com/paper/2206.07271/) (2022)
- [A Representative Study on Human Detection of Artificially Generated Media Across Countries](https://mlsanity.com/paper/2312.05976/) (2023)
- [Human Perception of LLM-generated Text Content in Social Media Environments](https://mlsanity.com/paper/2409.06653/) (2024)

## Built on by (most cited first, 2 of 2)

- [Benchmark of stylistic variation in LLM-generated texts](https://mlsanity.com/paper/2509.10179/) (2025)
- [Different Time, Different Language: Revisiting the Bias Against Non-Native Speakers in GPT Detectors](https://mlsanity.com/paper/2602.05769/) (2026)

## Abstract (arXiv)

This study investigates whether individuals can learn to accurately discriminate between human-written and AI-produced texts when provided with immediate feedback, and if they can use this feedback to recalibrate their self-perceived competence. We also explore the specific criteria individuals rely upon when making these decisions, focusing on textual style and perceived readability. We used GPT-4o to generate several hundred texts across various genres and text types comparable to Koditex, a multi-register corpus of human-written texts. We then presented randomized text pairs to 254 Czech native speakers who identified which text was human-written and which was AI-generated. Participants were randomly assigned to two conditions: one receiving immediate feedback after each trial, the other receiving no feedback until experiment completion. We recorded accuracy in identification, confidence levels, response times, and judgments about text readability along with demographic data and participants' engagement with AI technologies prior to the experiment. Participants receiving immediate feedback showed significant improvement in accuracy and confidence calibration. Participants initially held incorrect assumptions about AI-generated text features, including expectations about stylistic rigidity and readability. Notably, without feedback, participants made the most errors precisely when feeling most confident -- an issue largely resolved among the feedback group. The ability to differentiate between human and AI-generated texts can be effectively learned through targeted training with explicit feedback, which helps correct misconceptions about AI stylistic features and readability, as well as potential other variables that were not explored, while facilitating more accurate self-assessment. This finding might be particularly important in educational contexts.