# HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis

arXiv: https://arxiv.org/abs/2305.18226 · HTML: https://mlsanity.com/paper/2305.18226/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 26 May 2023.

## Builds on (most cited first, 16 of 16)

- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://mlsanity.com/paper/2301.11305/) (2023)
- [GLTR: Statistical Detection and Visualization of Generated Text](https://mlsanity.com/paper/1906.04043/) (2019)
- [Can AI-Generated Text be Reliably Detected?](https://mlsanity.com/paper/2303.11156/) (2023)
- [Evaluation of Text Generation: A Survey](https://mlsanity.com/paper/2006.14799/) (2020)
- [Automatic Detection of Generated Text is Easiest when Humans are Fooled](https://mlsanity.com/paper/1911.00650/) (2019)
- [Prompting PaLM for Translation: Assessing Strategies and Performance](https://mlsanity.com/paper/2211.09102/) (2022)
- [The Science of Detecting LLM-Generated Texts](https://mlsanity.com/paper/2303.07205/) (2023)
- [Unifying Human and Statistical Evaluation for Natural Language Generation](https://mlsanity.com/paper/1904.02792/) (2019)
- [On the Possibilities of AI-Generated Text Detection](https://mlsanity.com/paper/2304.04736/) (2023)
- [ChatGPT or Human? Detect and Explain. Explaining Decisions of Machine Learning Model for Detecting Short ChatGPT-generated Text](https://mlsanity.com/paper/2301.13852/) (2023)
- [Pre-train, Prompt and Recommendation: A Comprehensive Survey of Language Modelling Paradigm Adaptations in Recommender Systems](https://mlsanity.com/paper/2302.03735/) (2023)
- [Will ChatGPT get you caught? Rethinking of Plagiarism Detection](https://mlsanity.com/paper/2302.04335/) (2023)
- [Origin Tracing and Detecting of LLMs](https://mlsanity.com/paper/2304.14072/) (2023)
- [Practical Text Classification With Large Pre-Trained Language Models](https://mlsanity.com/paper/1812.01207/) (2018)
- [Perception, performance, and detectability of conversational artificial intelligence across 32 university courses](https://mlsanity.com/paper/2305.13934/) (2023)

## Built on by (most cited first, 30 of 30)

- [Spotting LLMs With Binoculars: Zero-Shot Detection of Machine-Generated Text](https://mlsanity.com/paper/2401.12070/) (2024)
- [Can LLM-Generated Misinformation Be Detected?](https://mlsanity.com/paper/2309.13788/) (2023)
- [A Survey of Text Watermarking in the Era of Large Language Models](https://mlsanity.com/paper/2312.07913/) (2023)
- [A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions](https://mlsanity.com/paper/2310.14724/) (2023)
- [MarkLLM: An Open-Source Toolkit for LLM Watermarking](https://mlsanity.com/paper/2405.10051/) (2024)
- [OUTFOX: LLM-Generated Essay Detection Through In-Context Learning with Adversarially Generated Examples](https://mlsanity.com/paper/2307.11729/) (2023)
- [Detecting Multimedia Generated by Large AI Models: A Survey](https://mlsanity.com/paper/2402.00045/) (2024)
- [A Survey on Detection of LLMs-Generated Content](https://mlsanity.com/paper/2310.15654/) (2023)
- [Mark My Words: Analyzing and Evaluating Language Model Watermarks](https://mlsanity.com/paper/2312.00273/) (2023)
- [DeTeCtive: Detecting AI-generated Text via Multi-Level Contrastive Learning](https://mlsanity.com/paper/2410.20964/) (2024)
- [Are We in the AI-Generated Text World Already? Quantifying and Monitoring AIGT on Social Media](https://mlsanity.com/paper/2412.18148/) (2024)
- [Who Wrote This? The Key to Zero-Shot LLM-Generated Text Detection Is GECScore](https://mlsanity.com/paper/2405.04286/) (2024)
- [Decoding the AI Pen: Techniques and Challenges in Detecting AI-Generated Text](https://mlsanity.com/paper/2403.05750/) (2024)
- [GenAI Content Detection Task 1: English and Multilingual Machine-Generated Text Detection: AI vs. Human](https://mlsanity.com/paper/2501.11012/) (2025)
- [AI "News" Content Farms Are Easy to Make and Hard to Detect: A Case Study in Italian](https://mlsanity.com/paper/2406.12128/) (2024)
- [Survey on AI-Generated Media Detection: From Non-MLLM to MLLM](https://mlsanity.com/paper/2502.05240/) (2025)
- [Does DetectGPT Fully Utilize Perturbation? Bridging Selective Perturbation to Fine-tuned Contrastive Learning Detector would be Better](https://mlsanity.com/paper/2402.00263/) (2024)
- [Is Contrasting All You Need? Contrastive Learning for the Detection and Attribution of AI-generated Text](https://mlsanity.com/paper/2407.09364/) (2024)
- [Detecting ChatGPT: A Survey of the State of Detecting ChatGPT-Generated Text](https://mlsanity.com/paper/2309.07689/) (2023)
- [Authorship Attribution in Multilingual Machine-Generated Texts](https://mlsanity.com/paper/2508.01656/) (2025)
- [AuthentiGPT: Detecting Machine-Generated Text via Black-Box Language Models Denoising](https://mlsanity.com/paper/2311.07700/) (2023)
- [A Training-free Method for LLM Text Attribution](https://mlsanity.com/paper/2501.02406/) (2025)
- [MOSAIC: Multiple Observers Spotting AI Content](https://mlsanity.com/paper/2409.07615/) (2024)
- [GigaCheck: Detecting LLM-generated Content via Object-Centric Span Localization](https://mlsanity.com/paper/2410.23728/) (2024)
- [Evaluating LLM-Contaminated Crowdsourcing Data Without Ground Truth](https://mlsanity.com/paper/2506.06991/) (2025)
- [PVMark: Enabling Public Verifiability for LLM Watermarking Schemes](https://mlsanity.com/paper/2510.26274/) (2025)
- [Different Time, Different Language: Revisiting the Bias Against Non-Native Speakers in GPT Detectors](https://mlsanity.com/paper/2602.05769/) (2026)
- [Detecting Machine-Generated Long-Form Content with Latent-Space Variables](https://mlsanity.com/paper/2410.03856/) (2024)
- [Breaking the Generator Barrier: Disentangled Representation for Generalizable AI-Text Detection](https://mlsanity.com/paper/2604.13692/) (2026)
- [Towards Inclusive Toxic Content Moderation: Addressing Vulnerabilities to Adversarial Attacks in Toxicity Classifiers Tackling LLM-generated Content](https://mlsanity.com/paper/2509.12672/) (2025)

## Abstract (arXiv)

As the use of Large Language Models (LLMs) in text generation tasks proliferates, concerns arise over their potential to compromise academic integrity. The education sector currently tussles with distinguishing student-authored homework assignments from AI-generated ones. This paper addresses the challenge by introducing HowkGPT, designed to identify homework assignments generated by AI. HowkGPT is built upon a dataset of academic assignments and accompanying metadata [17] and employs a pretrained LLM to compute perplexity scores for student-authored and ChatGPT-generated responses. These scores then assist in establishing a threshold for discerning the origin of a submitted assignment. Given the specificity and contextual nature of academic work, HowkGPT further refines its analysis by defining category-specific thresholds derived from the metadata, enhancing the precision of the detection. This study emphasizes the critical need for effective strategies to uphold academic integrity amidst the growing influence of LLMs and provides an approach to ensuring fair and accurate grading in educational institutions.