# JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correction

arXiv: https://arxiv.org/abs/2511.21700 · HTML: https://mlsanity.com/paper/2511.21700/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 30 references that are in the map.
- First posted on arXiv 16 Nov 2025.

## Builds on (most cited first, 11 of 11)

- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [FreeLB: Enhanced Adversarial Training for Natural Language Understanding](https://mlsanity.com/paper/1909.11764/) (2019)
- [GECToR -- Grammatical Error Correction: Tag, Not Rewrite](https://mlsanity.com/paper/2005.12592/) (2020)
- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://mlsanity.com/paper/2210.12484/) (2022)
- [Reaching Human-level Performance in Automatic Grammatical Error Correction: An Empirical Study](https://mlsanity.com/paper/1807.01270/) (2018)
- [Revisiting Grammatical Error Correction Evaluation and Beyond](https://mlsanity.com/paper/2211.01635/) (2022)
- [Automatic Metric Validation for Grammatical Error Correction](https://mlsanity.com/paper/1804.11225/) (2018)
- [Improving Seq2Seq Grammatical Error Correction via Decoding Interventions](https://mlsanity.com/paper/2310.14534/) (2023)
- [Efficient and Interpretable Grammatical Error Correction with Mixture of Experts](https://mlsanity.com/paper/2410.23507/) (2024)

## Abstract (arXiv)

Existing Grammatical Error Correction (GEC) systems suffer from limited reference diversity, leading to underestimated evaluation and restricted model generalization. To address this issue, we introduce the Judge of Edit-Level Validity (JELV), an automated framework to validate correction edits from grammaticality, faithfulness, and fluency. Using our proposed human-annotated Pair-wise Edit-level Validity Dataset (PEVData) as benchmark, JELV offers two implementations: a multi-turn LLM-as-Judges pipeline achieving 90% agreement with human annotators, and a distilled DeBERTa classifier with 85% precision on valid edits. We then apply JELV to reclassify misjudged false positives in evaluation and derive a comprehensive evaluation metric by integrating false positive decoupling and fluency scoring, resulting in state-of-the-art correlation with human judgments. We also apply JELV to filter LLM-generated correction candidates, expanding the BEA19's single-reference dataset containing 38,692 source sentences. Retraining top GEC systems on this expanded dataset yields measurable performance gains. JELV provides a scalable solution for enhancing reference diversity and strengthening both evaluation and model generalization.