# Where the Verifier Fails: A Category-Level Audit of Reward Signals in RLVR

arXiv: https://arxiv.org/abs/2609.01354 · HTML: https://mlsanity.com/paper/2609.01354/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 1 Sep 2026.

## Builds on (most cited first, 14 of 14)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluations](https://mlsanity.com/paper/2411.00640/) (2024)
- [Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models](https://mlsanity.com/paper/2502.17387/) (2025)
- [Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers](https://mlsanity.com/paper/2510.00915/) (2025)
- [Flaw or Artifact? Rethinking Prompt Sensitivity in Evaluating LLMs](https://mlsanity.com/paper/2509.01790/) (2025)
- [From Accuracy to Robustness: A Study of Rule- and Model-based Verifiers in Mathematical Reasoning](https://mlsanity.com/paper/2505.22203/) (2025)
- [METAL: Metamorphic Testing Framework for Analyzing Large-Language Model Qualities](https://mlsanity.com/paper/2312.06056/) (2023)
- [Faults in Our Formal Benchmarking: Dataset Defects and Evaluation Failures in Lean Theorem Proving](https://mlsanity.com/paper/2606.29493/) (2026)
- [Delay, Plateau, or Collapse: Evaluating the Impact of Systematic Verification Error on RLVR](https://mlsanity.com/paper/2605.02909/) (2026)
- [Auditing Reward Hackability in Code RL Training Environments](https://mlsanity.com/paper/2606.16062/) (2026)
- [When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR](https://mlsanity.com/paper/2607.11022/) (2026)
- [Beyond F1: Evaluating Coverage and Failure Recovery in AI Model Security Scanners](https://mlsanity.com/paper/2608.27424/) (2026)

## Built on by (most cited first, 3 of 3)

- [Are Verifier Errors Independent Within a GRPO Group? Evidence from Qwen2.5 Rollouts](https://mlsanity.com/paper/2609.06386/) (2026)
- [Re-executing Coding-Agent Evaluations: Unchanged Decisions and Unresolved Policy Risk](https://mlsanity.com/paper/2610.14460/) (2026)
- [Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVR](https://mlsanity.com/paper/2610.17824/) (2026)

## Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) and standard benchmark evaluation both rely on an automatic verifier that turns a free text answer into a binary reward. Prior work reports that one evaluation harness accepts only about 94% of its own ground truth answers, blaming LaTeX parsing. That is an aggregate: it does not say which answer forms consume the error budget. We supply the decomposition. We apply metamorphic testing to the verifier rather than the model, generating certified equivalent answer variants, that is, rewrites that preserve mathematical meaning by construction, so that any rejection is a provable false negative needing no human adjudication. We then measure rejection per answer category across four widely used verifiers over 307,420 verdicts. We find three things. (1) Self validation ranges from 53.8% to 95.2% on identical inputs, a spread of 41.3 points. The published figure describes one implementation, not the task; two configurations of the same library disagree on 49.9% of pairs. (2) The residual is not spread across parsing categories but concentrated in whitespace and punctuation, which account for 93.0% of in contract failures for the default LaTeX configuration. A trailing period or newline dominates the budget. (3) Separating rejection from execution failure shows that verifiers with similar aggregate error fail for opposite reasons, and that a reference numeric cascade accepts off by one wrong answers as a step function of magnitude, from 0% below 10^4 to 100% at or above, because its relative tolerance is scale invariant.