# Context is Key: Grammatical Error Detection with Contextual Word Representations

arXiv: https://arxiv.org/abs/1906.06593 · HTML: https://mlsanity.com/paper/1906.06593/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jun 2019.

## Built on by (most cited first, 14 of 14)

- [LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations](https://mlsanity.com/paper/2410.02707/) (2024)
- [Grammatical Error Correction: A Survey of the State of the Art](https://mlsanity.com/paper/2211.05166/) (2022)
- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [On the application of Large Language Models for language teaching and assessment technology](https://mlsanity.com/paper/2307.08393/) (2023)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [Automated scholarly paper review: Concepts, technologies, and challenges](https://mlsanity.com/paper/2111.07533/) (2021)
- [Zero-shot Sequence Labeling for Transformer-based Sentence Classifiers](https://mlsanity.com/paper/2103.14465/) (2021)
- [Exploring the Capacity of a Large-scale Masked Language Model to Recognize Grammatical Errors](https://mlsanity.com/paper/2108.12216/) (2021)
- [Oddballness: universal anomaly detection with language models](https://mlsanity.com/paper/2409.03046/) (2024)
- [Combining GCN and Transformer for Chinese Grammatical Error Detection](https://mlsanity.com/paper/2105.09085/) (2021)
- [Plug-Tagger: A Pluggable Sequence Labeling Framework Using Language Models](https://mlsanity.com/paper/2110.07331/) (2021)
- [Detection-Correction Structure via General Language Model for Grammatical Error Correction](https://mlsanity.com/paper/2405.17804/) (2024)
- [Zero-shot Cross-Lingual Transfer for Synthetic Data Generation in Grammatical Error Detection](https://mlsanity.com/paper/2407.11854/) (2024)
- [Generating Constructive Feedback on Stories via Reinforcement Learning](https://mlsanity.com/paper/2609.04824/) (2026)

## Abstract (arXiv)

Grammatical error detection (GED) in non-native writing requires systems to identify a wide range of errors in text written by language learners. Error detection as a purely supervised task can be challenging, as GED datasets are limited in size and the label distributions are highly imbalanced. Contextualized word representations offer a possible solution, as they can efficiently capture compositional information in language and can be optimized on large amounts of unsupervised data. In this paper, we perform a systematic comparison of ELMo, BERT and Flair embeddings (Peters et al., 2017; Devlin et al., 2018; Akbik et al., 2018) on a range of public GED datasets, and propose an approach to effectively integrate such representations in current methods, achieving a new state of the art on GED. We further analyze the strengths and weaknesses of different contextual embeddings for the task at hand, and present detailed analyses of their impact on different types of errors.