mlsanityData to 14 Oct 2026

Time Is Effort: Estimating Human Post-Editing Time for Grammar Error Correction Tool Evaluation

arXiv 2510.04394 ↗First posted 5 Oct 2025cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Text editing can involve several iterations of revision. Incorporating an efficient Grammar Error Correction (GEC) tool in the initial correction round can significantly impact further human editing effort and final text quality. This raises an interesting question to quantify GEC Tool usability: How much effort can the GEC Tool save users? We present the first large-scale dataset of post-editing (PE) time annotations and corrections for two English GEC test datasets (BEA19 and CoNLL14). We introduce Post-Editing Effort in Time (PEET) for GEC Tools as a human-focused evaluation scorer to rank any GEC Tool by estimating PE time-to-correct. Using our dataset, we quantify the amount of time saved by GEC Tools in text editing. Analyzing the edit type indicated that determining whether a sentence needs correction and edits like paraphrasing and punctuation changes had the greatest impact on PE time. Finally, comparison with human rankings shows that PEET correlates well with technical effort judgment, providing a new human-centric direction for evaluating GEC tool usability. We release our dataset and code at: https://github.com/ankitvad/PEET_Scorer.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 31 of its 106 references that are in the map. First posted on arXiv 5 Oct 2025.

Builds on

31 of 106 references.

References · 31

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  4. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  5. 2020AdapterFusion: Non-Destructive Task Composition for Transfer Learningcited by 788
  6. 2020AdapterHub: A Framework for Adapting Transformerscited by 444
  7. 2018Constituency Parsing with a Self-Attentive Encodercited by 199
  8. 2020GECToR -- Grammatical Error Correction: Tag, Not Rewritecited by 177
  9. 2021A Simple Recipe for Multilingual Grammatical Error Correctioncited by 108
  10. 2017JFLEG: A Fluency Corpus and Benchmark for Grammatical Error Correctioncited by 97
  11. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluationcited by 97
  12. 2019Parallel Iterative Edit Models for Local Sequence Transductioncited by 81
19 more
  1. 2023CoEdIT: Text Editing by Task-Specific Instruction Tuningcited by 68
  2. 2020Encoder-Decoder Models Can Benefit from Pre-trained Masked Language Models in Grammatical Error Correctioncited by 59
  3. 2019An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correctioncited by 55
  4. 2021Does syntax matter? A strong baseline for Aspect-based Sentiment Analysis with RoBERTacited by 41
  5. 2021LM-Critic: Language Models for Unsupervised Grammatical Error Correctioncited by 33
  6. 2018OpenNMT: Neural Machine Translation Toolkitcited by 27
  7. 2018Efficient Online Scalar Annotation with Bounded Supportcited by 27
  8. 2019A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learningcited by 26
  9. 2016GLEU Without Tuningcited by 24
  10. 2024Prompting open-source and commercial language models for grammatical error correction of English learner textcited by 24
  11. 2023Enhancing Grammatical Error Correction Systems with Explanationscited by 19
  12. 2022Revisiting Grammatical Error Correction Evaluation and Beyondcited by 18
  13. 2024GPT-3.5 for Grammatical Error Correctioncited by 18
  14. 2017The University of Edinburgh's Neural MT Systems for WMT17cited by 16
  15. 2020Personalizing Grammatical Error Correction: Adaptation to Proficiency Level and L1cited by 10
  16. 2016Phrase-based Machine Translation is State-of-the-Art for Automatic Grammatical Error Correctioncited by 8
  17. 2024Pillars of Grammatical Error Correction: Comprehensive Inspection Of Contemporary Approaches In The Era of Large Language Modelscited by 8
  18. 2022An Exploration of Post-Editing Effectiveness in Text Summarizationcited by 3
  19. 2025Corrections Meet Explanations: A Unified Framework for Explainable Grammatical Error Correctioncited by 1

Built on by

Not cited yet.