mlsanityData to 14 Oct 2026

Beyond Reference-Based Evaluation: Reward Models for Meta-Evaluation of Grammatical Error Correction

arXiv 2609.21231 ↗First posted 18 Sep 2026cs.CLcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Preference alignment for recommenders and RAG also fits: Human values and AI feedback alignment

Abstract (arXiv)

Reference-based metrics for Grammatical Error Correction (GEC) such as M2 and ERRANT assume that the reference set enumerates all valid edits, and therefore often penalize corrections that are grammatical and meaning-preserving but phrased differently. We introduce RM-EVAL, a reward model trained on human preference data from SEEDA, as a reference-free meta-evaluator that predicts human-like quality judgments at both full-sequence and partial-sequence levels. Beyond evaluation, we show that the same reward model can be used as a learning signal to improve GEC generation via Reward-Guided Text Generation (RGTG), which keeps a base GEC model frozen and performs online, reward-driven decoding. Across SEEDA, RM-EVAL achieves strong agreement with human rankings, and RGTG yields consistent gains in reward and external validation, demonstrating a unified framework for both assessing and enhancing GEC systems without relying on gold references.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 5 of its 10 references that are in the map. First posted on arXiv 18 Sep 2026.

Builds on

5 of 10 references.

References · 5

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2020Learning to summarize from human feedbackcited by 2,588
  3. 2024ARGS: Alignment as Reward-Guided Searchcited by 103
  4. 2024A Critical Look At Tokenwise Reward-Guided Text Generationcited by 12
  5. 2025Towards Cost-Effective Reward Guided Text Generationcited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Remedy: Learning Machine Translation Evaluation from Human Preferences with Reward Modeling
  2. 2024Improving Machine Translation with Human Feedback: An Exploration of Quality Estimation as a Reward Model
  3. 2024Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge
  4. 2024ChatLang-8: An LLM-Based Synthetic Data Generation Framework for Grammatical Error Correction
  5. 2026Listen, Correct, and Feed Back: Spoken Pedagogical Feedback Generation
  6. 2024Sample-Efficient Human Evaluation of Large Language Models via Maximum Discrepancy Competition
  7. 2023Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation
  8. 2023Fine-Grained Human Feedback Gives Better Rewards for Language Model Training