# Natural Language-based Assessment of L2 Oral Proficiency using LLMs

arXiv: https://arxiv.org/abs/2507.10200 · HTML: https://mlsanity.com/paper/2507.10200/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 33 references that are in the map.
- First posted on arXiv 14 Jul 2025.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Emergent Abilities of Large Language Models](https://mlsanity.com/paper/2206.07682/) (2022)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Models](https://mlsanity.com/paper/2307.07889/) (2023)
- [Proficiency assessment of L2 spoken English using wav2vec 2.0](https://mlsanity.com/paper/2210.13168/) (2022)
- [Can GPT-4 do L2 analytic assessment?](https://mlsanity.com/paper/2404.18557/) (2024)
- [Speak & Improve Corpus 2025: an L2 English Speech Corpus for Language Assessment and Feedback](https://mlsanity.com/paper/2412.11986/) (2024)
- [Speak & Improve Challenge 2025: Tasks and Baseline Systems](https://mlsanity.com/paper/2412.11985/) (2024)
- [Dolphin: A Spoken Language Proficiency Assessment System for Elementary Education](https://mlsanity.com/paper/1908.00358/) (2019)

## Built on by (most cited first, 6 of 6)

- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Listen, Correct, and Feed Back: Spoken Pedagogical Feedback Generation](https://mlsanity.com/paper/2604.14177/) (2026)
- [Towards Self-Referential Analytic Assessment: A Profile-Based Approach to L2 Writing Evaluation with LLMs](https://mlsanity.com/paper/2605.04298/) (2026)
- [Controlling Implicit Shortcut Reliance in L2 Spoken English Auto-markers](https://mlsanity.com/paper/2607.16085/) (2026)
- [CASA: Content-Acoustic Speaking Assessment with Speech Encoder and Large Language Model](https://mlsanity.com/paper/2608.13101/) (2026)
- [Interpretable, Fairly Evaluated Automated L2 Speaking Assessment that Beats the Single-Human Ceiling and Why Pause Encoding Does Not Change LLM Fluency Scores](https://mlsanity.com/paper/2608.26137/) (2026)

## Abstract (arXiv)

Natural language-based assessment (NLA) is an approach to second language assessment that uses instructions - expressed in the form of can-do descriptors - originally intended for human examiners, aiming to determine whether large language models (LLMs) can interpret and apply them in ways comparable to human assessment. In this work, we explore the use of such descriptors with an open-source LLM, Qwen 2.5 72B, to assess responses from the publicly available S&I Corpus in a zero-shot setting. Our results show that this approach - relying solely on textual information - achieves competitive performance: while it does not outperform state-of-the-art speech LLMs fine-tuned for the task, it surpasses a BERT-based model trained specifically for this purpose. NLA proves particularly effective in mismatched task settings, is generalisable to other data types and languages, and offers greater interpretability, as it is grounded in clearly explainable, widely applicable language descriptors.