# Can GPT-4 do L2 analytic assessment?

arXiv: https://arxiv.org/abs/2404.18557 · HTML: https://mlsanity.com/paper/2404.18557/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 60 references that are in the map.
- First posted on arXiv 29 Apr 2024.

## Builds on (most cited first, 8 of 8)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Automatic Text Scoring Using Neural Networks](https://mlsanity.com/paper/1606.04289/) (2016)
- [Language models and Automated Essay Scoring](https://mlsanity.com/paper/1909.09482/) (2019)

## Built on by (most cited first, 16 of 16)

- [Assessment of L2 Oral Proficiency using Speech Large Language Models](https://mlsanity.com/paper/2505.21148/) (2025)
- [LLMs can Perform Multi-Dimensional Analytic Writing Assessments: A Case Study of L2 Graduate-Level Academic English Writing](https://mlsanity.com/paper/2502.11368/) (2025)
- [LLMs in Education: Novel Perspectives, Challenges, and Opportunities](https://mlsanity.com/paper/2409.11917/) (2024)
- [Natural Language-based Assessment of L2 Oral Proficiency using LLMs](https://mlsanity.com/paper/2507.10200/) (2025)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Pronunciation Assessment with Multi-modal Large Language Models](https://mlsanity.com/paper/2407.09209/) (2024)
- [Exploring the Potential of Large Multimodal Models as Effective Alternatives for Pronunciation Assessment](https://mlsanity.com/paper/2503.11229/) (2025)
- [Exploiting the English Vocabulary Profile for L2 word-level vocabulary assessment with LLMs](https://mlsanity.com/paper/2506.02758/) (2025)
- [Grammatical Error Feedback: An Implicit Evaluation Approach](https://mlsanity.com/paper/2408.09565/) (2024)
- [Zero-Shot Grammar Competency Estimation Using Large Language Model Generated Pseudo Labels](https://mlsanity.com/paper/2511.13152/) (2025)
- [Towards interpretable models for language proficiency assessment: Predicting the CEFR level of Estonian learner texts](https://mlsanity.com/paper/2602.13102/) (2026)
- [From Prompting to Preference Optimization: A Comparative Study of LLM-based Automated Essay Scoring](https://mlsanity.com/paper/2603.06424/) (2026)
- [Exploiting the English Grammar Profile for L2 grammatical analysis with LLMs](https://mlsanity.com/paper/2603.17171/) (2026)
- [Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs](https://mlsanity.com/paper/2605.04298/) (2026)
- [Incentives Of EdTech: A Systematic Review Of EduNLP Research](https://mlsanity.com/paper/2606.13691/) (2026)
- [LLM Performance on a Real, Double-Marked GCSE Benchmark](https://mlsanity.com/paper/2606.24973/) (2026)

## Abstract (arXiv)

Automated essay scoring (AES) to evaluate second language (L2) proficiency has been a firmly established technology used in educational contexts for decades. Although holistic scoring has seen advancements in AES that match or even exceed human performance, analytic scoring still encounters issues as it inherits flaws and shortcomings from the human scoring process. The recent introduction of large language models presents new opportunities for automating the evaluation of specific aspects of L2 writing proficiency. In this paper, we perform a series of experiments using GPT-4 in a zero-shot fashion on a publicly available dataset annotated with holistic scores based on the Common European Framework of Reference and aim to extract detailed information about their underlying analytic components. We observe significant correlations between the automatically predicted analytic scores and multiple features associated with the individual proficiency components.