# The Great AI Witch Hunt: Reviewers Perception and (Mis)Conception of Generative AI in Research Writing

arXiv: https://arxiv.org/abs/2407.12015 · HTML: https://mlsanity.com/paper/2407.12015/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 91 references that are in the map.
- First posted on arXiv 27 Jun 2024.

## Builds on (most cited first, 16 of 16)

- [Survey of Hallucination in Natural Language Generation](https://mlsanity.com/paper/2202.03629/) (2022)
- [CoAuthor: Designing a Human-AI Collaborative Writing Dataset for Exploring Language Model Capabilities](https://mlsanity.com/paper/2201.06796/) (2022)
- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [Co-Writing with Opinionated Language Models Affects Users' Views](https://mlsanity.com/paper/2302.00560/) (2023)
- [Gmail Smart Compose: Real-Time Assisted Writing](https://mlsanity.com/paper/1906.00080/) (2019)
- [ChatGPT and a New Academic Reality: Artificial Intelligence-Written Research Papers and the Ethics of the Large Language Models in Scholarly Publishing](https://mlsanity.com/paper/2303.13367/) (2023)
- [The Who in XAI: How AI Background Shapes Perceptions of AI Explanations](https://mlsanity.com/paper/2107.13509/) (2021)
- [AI-Augmented Brainwriting: Investigating the use of LLMs in group ideation](https://mlsanity.com/paper/2402.14978/) (2024)
- [A Data-Driven Analysis of Workers' Earnings on Amazon Mechanical Turk](https://mlsanity.com/paper/1712.05796/) (2017)
- [The AI Ghostwriter Effect: When Users Do Not Perceive Ownership of AI-Generated Text But Self-Declare as Authors](https://mlsanity.com/paper/2303.03283/) (2023)
- [Rambler: Supporting Writing With Speech via LLM-Assisted Gist Manipulation](https://mlsanity.com/paper/2401.10838/) (2024)
- [Assigning AI: Seven Approaches for Students, with Prompts](https://mlsanity.com/paper/2306.10052/) (2023)
- [The Dearth of the Author in AI-Supported Writing](https://mlsanity.com/paper/2404.10289/) (2024)
- [Sora OpenAI's Prelude: Social Media Perspectives on Sora OpenAI and the Future of AI Video Generation](https://mlsanity.com/paper/2403.14665/) (2024)
- [Large Scale Analysis of Multitasking Behavior During Remote Meetings](https://mlsanity.com/paper/2101.11865/) (2021)
- [Augmenting the Author: Exploring the Potential of AI Collaboration in Academic Writing](https://mlsanity.com/paper/2404.16071/) (2024)

## Built on by (most cited first, 6 of 6)

- [What happens when reviewers receive AI feedback in their reviews?](https://mlsanity.com/paper/2602.13817/) (2026)
- [The Emerging AI Paper-Review Arms Race: Adversarial Co-Evolution in Scholarly Publishing](https://mlsanity.com/paper/2609.07713/) (2026)
- [Who is Responsible When AI Fails? Mapping Causes, Entities, and Consequences of AI Privacy and Ethical Incidents](https://mlsanity.com/paper/2504.01029/) (2025)
- [AI-Generated Content in Cross-Domain Applications: Research Trends, Challenges and Propositions](https://mlsanity.com/paper/2509.11151/) (2025)
- [LLM or Human? Perceptions of Trust and Information Quality in Research Summaries](https://mlsanity.com/paper/2601.15556/) (2026)
- [Magic, Madness, Heaven, Sin: LLM Output Diversity is Everything, Everywhere, All at Once](https://mlsanity.com/paper/2604.01504/) (2026)

## Abstract (arXiv)

Generative AI (GenAI) use in research writing is growing fast. However, it is unclear how peer reviewers recognize or misjudge AI-augmented manuscripts. To investigate the impact of AI-augmented writing on peer reviews, we conducted a snippet-based online survey with 17 peer reviewers from top-tier HCI conferences. Our findings indicate that while AI-augmented writing improves readability, language diversity, and informativeness, it often lacks research details and reflective insights from authors. Reviewers consistently struggled to distinguish between human and AI-augmented writing but their judgements remained consistent. They noted the loss of a "human touch" and subjective expressions in AI-augmented writing. Based on our findings, we advocate for reviewer guidelines that promote impartial evaluations of submissions, regardless of any personal biases towards GenAI. The quality of the research itself should remain a priority in reviews, regardless of any preconceived notions about the tools used to create it. We emphasize that researchers must maintain their authorship and control over the writing process, even when using GenAI's assistance.