# The Crutch or the Ceiling? How Different Generations of LLMs Shape EFL Student Writings

arXiv: https://arxiv.org/abs/2604.15460 · HTML: https://mlsanity.com/paper/2604.15460/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 79 references that are in the map.
- First posted on arXiv 16 Apr 2026.

## Builds on (most cited first, 22 of 22)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Why Language Models Hallucinate](https://mlsanity.com/paper/2509.04664/) (2025)
- [Understanding Catastrophic Forgetting in Language Models via Implicit Inference](https://mlsanity.com/paper/2309.10105/) (2023)
- [Your Brain on ChatGPT: Accumulation of Cognitive Debt when Using an AI Assistant for Essay Writing Task](https://mlsanity.com/paper/2506.08872/) (2025)
- [Text Readability Assessment for Second Language Learners](https://mlsanity.com/paper/1906.07580/) (2019)
- [LLM-as-a-tutor in EFL Writing Education: Focusing on Evaluation of Student-LLM Interaction](https://mlsanity.com/paper/2310.05191/) (2023)
- [Common Corpus: The Largest Collection of Ethical Data for LLM Pre-Training](https://mlsanity.com/paper/2506.01732/) (2025)
- [The Cambridge Law Corpus: A Dataset for Legal AI Research](https://mlsanity.com/paper/2309.12269/) (2023)
- [LLMs as Writing Assistants: Exploring Perspectives on Sense of Ownership and Reasoning](https://mlsanity.com/paper/2404.00027/) (2024)
- [Script&Shift: A Layered Interface Paradigm for Integrating Content Development and Rhetorical Strategy with LLM Writing Assistants](https://mlsanity.com/paper/2502.10638/) (2025)
- [Help Me Write a Story: Evaluating LLMs' Ability to Generate Writing Feedback](https://mlsanity.com/paper/2507.16007/) (2025)
- [MedReadMe: A Systematic Study for Fine-grained Sentence Readability in Medical Domain](https://mlsanity.com/paper/2405.02144/) (2024)
- [The Role of AI in Human-AI Creative Writing for Hong Kong Secondary Students](https://mlsanity.com/paper/2304.11276/) (2023)
- [LLM-Driven Learning Analytics Dashboard for Teachers in EFL Writing Education](https://mlsanity.com/paper/2410.15025/) (2024)
- [How Large Language Models Are Changing MOOC Essay Answers: A Comparison of Pre- and Post-LLM Responses](https://mlsanity.com/paper/2504.13038/) (2025)
- [Exploring AI-Generated Text in Student Writing: How Does AI Help?](https://mlsanity.com/paper/2304.02478/) (2023)
- [Towards Full Authorship with AI: Supporting Revision with AI-Generated Views](https://mlsanity.com/paper/2403.01055/) (2024)
- [An Empirical Study to Understand How Students Use ChatGPT for Writing Essays](https://mlsanity.com/paper/2501.10551/) (2025)
- [A Transfer Learning Based Model for Text Readability Assessment in German](https://mlsanity.com/paper/2207.06265/) (2022)
- [Comparing human and LLM proofreading in L2 writing: Impact on lexical and syntactic features](https://mlsanity.com/paper/2506.09021/) (2025)
- [Approaching the Limits to EFL Writing Enhancement with AI-generated Text and Diverse Learners](https://mlsanity.com/paper/2503.00367/) (2025)

## Abstract (arXiv)

The rapid evolution of Large Language Models (LLMs) has made them powerful tools for enhancing student writing. This study explores the extent and limitations of LLMs in assisting secondary-level English as a Foreign Language (EFL) students with their writing tasks. While existing studies focus on output quality, our research examines the developmental shift in LLMs and their impact on EFL students, assessing whether smarter models act as true scaffolds or mere compensatory crutches. To achieve this, we analyse student compositions assisted by LLMs before and after ChatGPT's release, using both expert qualitative scoring and quantitative metrics (readability tests, Pearson's correlation coefficient, MTLD, and others). Our results indicate that advanced LLMs boost assessment scores and lexical diversity for lower-proficiency learners, potentially masking their true ability. Crucially, increased LLM assistance correlated negatively with human expert ratings, suggesting surface fluency without deep coherence. To transform AI-assisted practice into genuine learning, pedagogy must shift from focusing on output quality to verifying the learning process. Educators should align AI functions, specifically differentiating ideational scaffolding from textual production, within the learner's Zone of Proximal Development.