mlsanityData to 14 Oct 2026

Automatic Metric Validation for Grammatical Error Correction

arXiv 1804.11225 ↗First posted 30 Apr 2018cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

Metric validation in Grammatical Error Correction (GEC) is currently done by observing the correlation between human and metric-induced rankings. However, such correlation studies are costly, methodologically troublesome, and suffer from low inter-rater agreement. We propose MAEGE, an automatic methodology for GEC metric validation, that overcomes many of the difficulties with existing practices. Experiments with maege shed a new light on metric quality, showing for example that the standard M2 metric fares poorly on corpus-level ranking. Moreover, we use MAEGE to perform a detailed analysis of metric behavior, showing that correcting some types of errors is consistently penalized by existing metrics.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Apr 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 5; 2023: 3; 2024: 2; 2025: 3; 2026: 1.

17 papers.

Sort:

  1. 2023Efficient Benchmarking of Language Modelscited by 71
  2. 2021Perturbation CheckLists for Evaluating NLG Evaluation Metricscited by 50
  3. 2022Grammatical Error Correction: A Survey of the State of the Artcited by 43
  4. 2022On the Evaluation Metrics for Paraphrase Generationcited by 32
  5. 2022EditEval: An Instruction-Based Benchmark for Text Improvementscited by 30
  6. 2024CASIMIR: A Corpus of Scientific Articles enhanced with Multiple Author-Integrated Revisionscited by 14
  7. 2022Towards standardizing Korean Grammatical Error Correction: Datasets and Annotationcited by 13
  8. 2021SERRANT: a syntactic classifier for English Grammatical Error Typescited by 6
  9. 2023CLEME: Debiasing Multi-reference Evaluation for Grammatical Error Correctioncited by 5
  10. 2021ComSum: Commit Messages Summarization and Meaning Preservationcited by 3
  11. 2024Revisiting Meta-evaluation for Grammatical Error Correctioncited by 3
  12. 2025IMPARA-GED: Grammatical Error Detection is Boosting Reference-free Grammatical Error Quality Estimatorcited by 2
5 more
  1. 2022Some Grammatical Errors are Frequent, Others are Importantcited by 0
  2. 2023MuLER: Detailed and Scalable Reference-based Evaluationcited by 0
  3. 2025JELV: A Judge of Edit-Level Validity for Evaluation and Automated Reference Expansion in Grammatical Error Correctioncited by 0
  4. 2025Minimal-Edit Instruction Tuning for Low-Resource Indic GECcited by 0
  5. 2026Does Linguistic Structure Enrichment Enhance Coherence Assessment? Not With Current Architecturescited by 0