# On the application of Large Language Models for language teaching and assessment technology

arXiv: https://arxiv.org/abs/2307.08393 · HTML: https://mlsanity.com/paper/2307.08393/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 57 of its 126 references that are in the map.
- First posted on arXiv 17 Jul 2023.

## Builds on (most cited first, 30 of 57)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [The Pile: An 800GB Dataset of Diverse Text for Language Modeling](https://mlsanity.com/paper/2101.00027/) (2020)
- [Extracting Training Data from Large Language Models](https://mlsanity.com/paper/2012.07805/) (2020)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Energy and Policy Considerations for Deep Learning in NLP](https://mlsanity.com/paper/1906.02243/) (2019)
- [PaLM 2 Technical Report](https://mlsanity.com/paper/2305.10403/) (2023)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [RACE: Large-scale ReAding Comprehension Dataset From Examinations](https://mlsanity.com/paper/1704.04683/) (2017)
- [GLM-130B: An Open Bilingual Pre-trained Model](https://mlsanity.com/paper/2210.02414/) (2022)
- [SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models](https://mlsanity.com/paper/2303.08896/) (2023)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Check Your Facts and Try Again: Improving Large Language Models with External Knowledge and Automated Feedback](https://mlsanity.com/paper/2302.12813/) (2023)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cycle](https://mlsanity.com/paper/1901.10002/) (2019)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [Measuring the Carbon Intensity of AI in Cloud Instances](https://mlsanity.com/paper/2206.05229/) (2022)
- [Debiasing Pre-trained Contextualised Embeddings](https://mlsanity.com/paper/2101.09523/) (2021)
- [Towards an Automatic Turing Test: Learning to Evaluate Dialogue Responses](https://mlsanity.com/paper/1708.07149/) (2017)

## Built on by (most cited first, 27 of 27)

- [Knowledge Conflicts for LLMs: A Survey](https://mlsanity.com/paper/2403.08319/) (2024)
- [The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models](https://mlsanity.com/paper/2406.05761/) (2024)
- [Fine-tuning ChatGPT for Automatic Scoring](https://mlsanity.com/paper/2310.10072/) (2023)
- [Retrieval-augmented Generation to Improve Math Question-Answering: Trade-offs Between Groundedness and Human Preference](https://mlsanity.com/paper/2310.03184/) (2023)
- [Prompting open-source and commercial language models for grammatical error correction of English learner text](https://mlsanity.com/paper/2401.07702/) (2024)
- [From Text to Transformation: A Comprehensive Review of Large Language Models' Versatility](https://mlsanity.com/paper/2402.16142/) (2024)
- [A Novel Approach to Scalable and Automatic Topic-Controlled Question Generation in Education](https://mlsanity.com/paper/2501.05220/) (2025)
- [Beyond Agreement: Diagnosing the Rationale Alignment of Automated Essay Scoring Methods based on Linguistically-informed Counterfactuals](https://mlsanity.com/paper/2405.19433/) (2024)
- [ELLMA-T: an Embodied LLM-agent for Supporting English Language Learning in Social VR](https://mlsanity.com/paper/2410.02406/) (2024)
- [Exploring the Potential of Large Language Models for Improving Digital Forensic Investigation Efficiency](https://mlsanity.com/paper/2402.19366/) (2024)
- [Generating AI Literacy MCQs: A Multi-Agent LLM Approach](https://mlsanity.com/paper/2412.00970/) (2024)
- [Opportunities and Challenges of LLMs in Education: An NLP Perspective](https://mlsanity.com/paper/2507.22753/) (2025)
- [Can LLMs Grade Short-Answer Reading Comprehension Questions : An Empirical Study with a Novel Dataset](https://mlsanity.com/paper/2310.18373/) (2023)
- [Toward Beginner-Friendly LLMs for Language Learning: Controlling Difficulty in Conversation](https://mlsanity.com/paper/2506.04072/) (2025)
- [The Use of Multiple Conversational Agent Interlocutors in Learning](https://mlsanity.com/paper/2312.16534/) (2023)
- [An overview of artificial intelligence in computer-assisted language learning](https://mlsanity.com/paper/2505.02032/) (2025)
- [Potential Societal Biases of ChatGPT in Higher Education: A Scoping Review](https://mlsanity.com/paper/2311.14381/) (2023)
- [Future-proofing Education: A Prototype for Simulating Oral Examinations Using Large Language Models](https://mlsanity.com/paper/2401.06160/) (2023)
- [Detection-Correction Structure via General Language Model for Grammatical Error Correction](https://mlsanity.com/paper/2405.17804/) (2024)
- [Enhancing LLM Character-Level Manipulation via Divide and Conquer](https://mlsanity.com/paper/2502.08180/) (2025)
- [Explanation based In-Context Demonstrations Retrieval for Multilingual Grammatical Error Correction](https://mlsanity.com/paper/2502.08507/) (2025)
- [On the effectiveness of LLMs for automatic grading of open-ended questions in Spanish](https://mlsanity.com/paper/2503.18072/) (2025)
- [Superstudent intelligence in thermodynamics](https://mlsanity.com/paper/2506.09822/) (2025)
- [Safe Generative Chats in a WhatsApp Intelligent Tutoring System](https://mlsanity.com/paper/2407.04915/) (2024)
- [Incentives Of EdTech: A Systematic Review Of EduNLP Research](https://mlsanity.com/paper/2606.13691/) (2026)
- [Reducing Barriers to Academic Support: Evaluating a Course-Specific RAG System for Addressing Help-Seeking Disparities in Higher Education](https://mlsanity.com/paper/2609.21600/) (2026)
- [Which LLM to pick? Online Active Model Selection for Large Language Models](https://mlsanity.com/paper/2610.01592/) (2026)

## Abstract (arXiv)

The recent release of very large language models such as PaLM and GPT-4 has made an unprecedented impact in the popular media and public consciousness, giving rise to a mixture of excitement and fear as to their capabilities and potential uses, and shining a light on natural language processing research which had not previously received so much attention. The developments offer great promise for education technology, and in this paper we look specifically at the potential for incorporating large language models in AI-driven language teaching and assessment systems. We consider several research areas and also discuss the risks and ethical considerations surrounding generative AI in education technology for language learners. Overall we find that larger language models offer improvements over previous models in text generation, opening up routes toward content generation which had not previously been plausible. For text generation they must be prompted carefully and their outputs may need to be reshaped before they are ready for use. For automated grading and grammatical error correction, tasks whose progress is checked on well-known benchmarks, early investigations indicate that large language models on their own do not improve on state-of-the-art results according to standard evaluation metrics. For grading it appears that linguistic features established in the literature should still be used for best performance, and for error correction it may be that the models can offer alternative feedback styles which are not measured sensitively with existing methods. In all cases, there is work to be done to experiment with the inclusion of large language models in education technology for language learners, in order to properly understand and report on their capacities and limitations, and to ensure that foreseeable risks such as misinformation and harmful bias are mitigated.