# How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses

arXiv: https://arxiv.org/abs/2504.13038 · HTML: https://mlsanity.com/paper/2504.13038/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 35 references that are in the map.
- First posted on arXiv 17 Apr 2025.

## Builds on (most cited first, 10 of 10)

- [Mapping the Increasing Use of LLMs in Scientific Papers](https://mlsanity.com/paper/2404.01268/) (2024)
- [Delving into LLM-assisted writing in biomedical publications through excess vocabulary](https://mlsanity.com/paper/2406.07016/) (2024)
- [Beware of Metacognitive Laziness: Effects of Generative Artificial Intelligence on Learning Motivation, Processes, and Performance](https://mlsanity.com/paper/2412.09315/) (2024)
- [Towards Understanding the Interplay of Generative Artificial Intelligence and the Internet](https://mlsanity.com/paper/2306.06130/) (2023)
- [Is ChatGPT Transforming Academics' Writing Style?](https://mlsanity.com/paper/2404.08627/) (2024)
- [Classification of Human- and AI-Generated Texts: Investigating Features for ChatGPT](https://mlsanity.com/paper/2308.05341/) (2023)
- [Can We Trust AI-Generated Educational Content? Comparative Analysis of Human and AI-Generated Learning Resources](https://mlsanity.com/paper/2306.10509/) (2023)
- [Playing with words: Comparing the vocabulary and lexical diversity of ChatGPT and humans](https://mlsanity.com/paper/2308.07462/) (2023)
- ["Genlangs" and Zipf's Law: Do languages generated by ChatGPT statistically look human?](https://mlsanity.com/paper/2304.12191/) (2023)
- [The Language Labyrinth: Constructive Critique on the Terminology Used in the AI Discourse](https://mlsanity.com/paper/2307.10292/) (2023)

## Built on by (most cited first, 5 of 5)

- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [On the Detectability of LLM-Generated Text: What Exactly Is LLM-Generated Text?](https://mlsanity.com/paper/2510.20810/) (2025)
- [The Variance Paradox: How AI Reduces Diversity but Increases Novelty](https://mlsanity.com/paper/2508.19264/) (2025)
- [The Crutch or the Ceiling? How Different Generations of LLMs Shape EFL Student Writings](https://mlsanity.com/paper/2604.15460/) (2026)
- [Measuring Changes in Instructor Class Design and Student Learning After the Release of Large Language Models (LLMs)](https://mlsanity.com/paper/2605.16284/) (2026)

## Abstract (arXiv)

The release of ChatGPT in late 2022 caused a flurry of activity and concern in the academic and educational communities. Some see the tool's ability to generate human-like text that passes at least cursory inspections for factual accuracy ``often enough'' a golden age of information retrieval and computer-assisted learning. Some, on the other hand, worry the tool may lead to unprecedented levels of academic dishonesty and cheating. In this work, we quantify some of the effects of the emergence of Large Language Models (LLMs) on online education by analyzing a multi-year dataset of student essay responses from a free university-level MOOC on AI ethics. Our dataset includes essays submitted both before and after ChatGPT's release. We find that the launch of ChatGPT coincided with significant changes in both the length and style of student essays, mirroring observations in other contexts such as academic publishing. We also observe -- as expected based on related public discourse -- changes in prevalence of key content words related to AI and LLMs, but not necessarily the general themes or topics discussed in the student essays as identified through (dynamic) topic modeling.