# Game of Tones: Faculty detection of GPT-4 generated content in university assessments

arXiv: https://arxiv.org/abs/2305.18081 · HTML: https://mlsanity.com/paper/2305.18081/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 80 references that are in the map.
- First posted on arXiv 29 May 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [The Internal State of an LLM Knows When It's Lying](https://mlsanity.com/paper/2304.13734/) (2023)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Text](https://mlsanity.com/paper/2107.00061/) (2021)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [GPT detectors are biased against non-native English writers](https://mlsanity.com/paper/2304.02819/) (2023)
- [On the Possibilities of AI-Generated Text Detection](https://mlsanity.com/paper/2304.04736/) (2023)
- [Eight Things to Know about Large Language Models](https://mlsanity.com/paper/2304.00612/) (2023)
- [On the Educational Impact of ChatGPT: Is Artificial Intelligence Ready to Obtain a University Degree?](https://mlsanity.com/paper/2303.11146/) (2023)
- [Fooling MOSS Detection with Pretrained Language Models](https://mlsanity.com/paper/2201.07406/) (2022)

## Built on by (most cited first, 16 of 16)

- [People cannot distinguish GPT-4 from a human in a Turing test](https://mlsanity.com/paper/2405.08007/) (2024)
- [Zero-Shot Detection of Machine-Generated Codes](https://mlsanity.com/paper/2310.05103/) (2023)
- [Text Generation: A Systematic Literature Review of Tasks, Evaluation, and Challenges](https://mlsanity.com/paper/2405.15604/) (2024)
- [GenAI Detection Tools, Adversarial Techniques and Implications for Inclusivity in Higher Education](https://mlsanity.com/paper/2403.19148/) (2024)
- [The AI Assessment Scale (AIAS): A Framework for Ethical Integration of Generative AI in Educational Assessment](https://mlsanity.com/paper/2312.07086/) (2023)
- [GPT-4 is judged more human than humans in displaced and inverted Turing tests](https://mlsanity.com/paper/2407.08853/) (2024)
- [The AI Assessment Scale (AIAS) in action: A pilot implementation of GenAI supported assessment- A Preprint](https://mlsanity.com/paper/2403.14692/) (2024)
- [Deepfakes and Higher Education: A Research Agenda and Scoping Review of Synthetic Media](https://mlsanity.com/paper/2404.15601/) (2024)
- [Understanding Student and Academic Staff Perceptions of AI Use in Assessment and Feedback](https://mlsanity.com/paper/2406.15808/) (2024)
- [Generative AI Tools in Academic Research: Applications and Implications for Qualitative and Quantitative Research Methodologies](https://mlsanity.com/paper/2408.06872/) (2024)
- [Generative AI and Agency in Education: A Critical Scoping Review and Thematic Analysis](https://mlsanity.com/paper/2411.00631/) (2024)
- [The EAP-AIAS: Adapting the AI Assessment Scale for English for Academic Purposes](https://mlsanity.com/paper/2408.01075/) (2024)
- [A Peek Behind the Curtain: Using Step-Around Prompt Engineering to Identify Bias and Misinformation in GenAI Models](https://mlsanity.com/paper/2503.15205/) (2025)
- [Methodological Variation in Studying Staff and Student Perceptions of AI](https://mlsanity.com/paper/2602.11158/) (2025)
- [The Misclassification of Autistic Writing as AI-Generated](https://mlsanity.com/paper/2607.14729/) (2026)
- [Evaluating the Semantic-to-Geometric Gap in Adversarial Defenses Against Vision-Language Model-Based Plagiarism](https://mlsanity.com/paper/2609.26733/) (2026)

## Abstract (arXiv)

This study explores the robustness of university assessments against the use of Open AI's Generative Pre-Trained Transformer 4 (GPT-4) generated content and evaluates the ability of academic staff to detect its use when supported by the Turnitin Artificial Intelligence (AI) detection tool. The research involved twenty-two GPT-4 generated submissions being created and included in the assessment process to be marked by fifteen different faculty members. The study reveals that although the detection tool identified 91% of the experimental submissions as containing some AI-generated content, the total detected content was only 54.8%. This suggests that the use of adversarial techniques regarding prompt engineering is an effective method in evading AI detection tools and highlights that improvements to AI detection software are needed. Using the Turnitin AI detect tool, faculty reported 54.5% of the experimental submissions to the academic misconduct process, suggesting the need for increased awareness and training into these tools. Genuine submissions received a mean score of 54.4, whereas AI-generated content scored 52.3, indicating the comparable performance of GPT-4 in real-life situations. Recommendations include adjusting assessment strategies to make them more resistant to the use of AI tools, using AI-inclusive assessment where possible, and providing comprehensive training programs for faculty and students. This research contributes to understanding the relationship between AI-generated content and academic assessment, urging further investigation to preserve academic integrity.