# Beyond Reference-Based Evaluation: Reward Models for Meta-Evaluation of Grammatical Error Correction

arXiv: https://arxiv.org/abs/2609.21231 · HTML: https://mlsanity.com/paper/2609.21231/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Preference alignment for recommenders and RAG, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 5 of its 10 references that are in the map.
- First posted on arXiv 18 Sep 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Preference alignment for recommenders and RAG](https://mlsanity.com/alignment/preference-alignment-for-recommenders-and-rag/)

## Builds on (most cited first, 5 of 5)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [ARGS: Alignment as Reward-Guided Search](https://mlsanity.com/paper/2402.01694/) (2024)
- [A Critical Look At Tokenwise Reward-Guided Text Generation](https://mlsanity.com/paper/2406.07780/) (2024)
- [Towards Cost-Effective Reward Guided Text Generation](https://mlsanity.com/paper/2502.04517/) (2025)

## Abstract (arXiv)

Reference-based metrics for Grammatical Error Correction (GEC) such as M2 and ERRANT assume that the reference set enumerates all valid edits, and therefore often penalize corrections that are grammatical and meaning-preserving but phrased differently. We introduce RM-EVAL, a reward model trained on human preference data from SEEDA, as a reference-free meta-evaluator that predicts human-like quality judgments at both full-sequence and partial-sequence levels. Beyond evaluation, we show that the same reward model can be used as a learning signal to improve GEC generation via Reward-Guided Text Generation (RGTG), which keeps a base GEC model frozen and performs online, reward-driven decoding. Across SEEDA, RM-EVAL achieves strong agreement with human rankings, and RGTG yields consistent gains in reward and external validation, demonstrating a unified framework for both assessing and enhancing GEC systems without relying on gold references.