# Enhancing Grammatical Error Detection using BERT with Cleaned Lang-8 Dataset

arXiv: https://arxiv.org/abs/2411.15523 · HTML: https://mlsanity.com/paper/2411.15523/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 20 references that are in the map.
- First posted on arXiv 23 Nov 2024.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Advancements in Arabic Grammatical Error Detection and Correction: An Empirical Investigation](https://mlsanity.com/paper/2305.14734/) (2023)
- [Semi-supervised Multitask Learning for Sequence Labeling](https://mlsanity.com/paper/1704.07156/) (2017)
- [Wronging a Right: Generating Better Errors to Improve Grammatical Error Detection](https://mlsanity.com/paper/1810.00668/) (2018)
- [Neural Quality Estimation with Multiple Hypotheses for Grammatical Error Correction](https://mlsanity.com/paper/2105.04443/) (2021)
- [Jointly Learning to Label Sentences and Tokens](https://mlsanity.com/paper/1811.05949/) (2018)
- [Probing for targeted syntactic knowledge through grammatical error detection](https://mlsanity.com/paper/2210.16228/) (2022)
- [Bangla Grammatical Error Detection Using T5 Transformer Model](https://mlsanity.com/paper/2303.10612/) (2023)
- [Evaluation of really good grammatical error correction](https://mlsanity.com/paper/2308.08982/) (2023)

## Abstract (arXiv)

This paper presents an improved LLM based model for Grammatical Error Detection (GED), which is a very challenging and equally important problem for many applications. The traditional approach to GED involved hand-designed features, but recently, Neural Networks (NN) have automated the discovery of these features, improving performance in GED. Traditional rule-based systems have an F1 score of 0.50-0.60 and earlier machine learning models give an F1 score of 0.65-0.75, including decision trees and simple neural networks. Previous deep learning models, for example, Bi-LSTM, have reported F1 scores within the range from 0.80 to 0.90. In our study, we have fine-tuned various transformer models using the Lang8 dataset rigorously cleaned by us. In our experiments, the BERT-base-uncased model gave an impressive performance with an F1 score of 0.91 and accuracy of 98.49% on training data and 90.53% on testing data, also showcasing the importance of data cleaning. Increasing model size using BERT-large-uncased or RoBERTa-large did not give any noticeable improvements in performance or advantage for this task, underscoring that larger models are not always better. Our results clearly show how far rigorous data cleaning and simple transformer-based models can go toward significantly improving the quality of GED.