# Multilingual Verifier Bias in RLVR: Benchmark, Rollout Diagnosis, and the Cross-Lingual Selection Bottleneck

arXiv: https://arxiv.org/abs/2608.20362 · HTML: https://mlsanity.com/paper/2608.20362/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning.
- Builds on 15 of its 17 references that are in the map.
- First posted on arXiv 17 Jun 2026.

## Placed in

- LLM reasoning / [Reinforcement learning for reasoning](https://mlsanity.com/lr/reinforcement-learning-for-reasoning/)

## Builds on (most cited first, 15 of 15)

- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Language Models are Multilingual Chain-of-Thought Reasoners](https://mlsanity.com/paper/2210.03057/) (2022)
- [Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifiers](https://mlsanity.com/paper/2510.00915/) (2025)
- [How Reliable is Multilingual LLM-as-a-Judge?](https://mlsanity.com/paper/2505.12201/) (2025)
- [LLMs Gaming Verifiers: RLVR can Lead to Reward Hacking](https://mlsanity.com/paper/2604.15149/) (2026)
- [TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoning](https://mlsanity.com/paper/2505.14625/) (2025)
- [NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning](https://mlsanity.com/paper/2505.16022/) (2025)
- [Noisy Data is Destructive to Reinforcement Learning with Verifiable Rewards](https://mlsanity.com/paper/2603.16140/) (2026)
- [Verifier-Free RL for LLMs via Intrinsic Gradient-Norm Reward](https://mlsanity.com/paper/2605.09920/) (2026)
- [Cross-lingual Self-Consistency for Multilingual Reasoning with Language Models](https://mlsanity.com/paper/2606.01464/) (2026)
- [Cross-Lingual Consistency: A Novel Inference Framework for Advancing Reasoning in Large Language Models](https://mlsanity.com/paper/2504.01857/) (2025)
- [MATH-PT: A Math Reasoning Benchmark for European and Brazilian Portuguese](https://mlsanity.com/paper/2604.25926/) (2026)
- [Soft-SVeRL: Self-Verified Reinforcement Learning with Soft Rewards](https://mlsanity.com/paper/2605.28561/) (2026)

## Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) is a standard recipe for training large language models on mathematical reasoning, where an answer verifier serves as a language-neutral reward function. We show that this assumption fails in multilingual settings: an exact-match verifier turns format and script variation into language-dependent false-negative reward noise. We introduce a reusable protocol for auditing multilingual RLVR rewards: a verifier-robustness suite, a rollout-diagnosis procedure, and language-conditioned reward-error metrics for Japanese, English, and Chinese answers. On MGSM rollouts with k=8, the exact-match proxy rejects trusted-correct answers at sharply different rates by language across Qwen3-4B, Qwen3-8B, and Llama-3.1-8B-Instruct; for Qwen3-8B, the false-negative rate reaches 0.642 on JP against 0.122 on EN and 0.073 on CN. A plain-numeric probe localizes the mechanism to the final-answer interface: an interface model drives reward-error VLB to zero while the residual accuracy gap is unchanged. We then expose a cross-lingual selection bottleneck: on MGSM250 rollouts, a target-local aggregation rule using no trusted labels closes 55-78% of the average selection gap, and over 95% of repairs require genuine cross-lingual support. The bottleneck replicates on a 483-problem MATH-500 set. A controlled training audit shows that rule-GRPO raises trusted accuracy while the reward-error VLB stays high. The unifying message is operational: multilingual RLVR rewards should be audited by language and by answer interface before they are optimized.