mlsanityData to 7 Oct 2026

Consensus Group Relative Policy Optimization for Text Generation

arXiv 2602.03102 ↗First posted 3 Feb 2026cs.LGcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Many strong decoding methods for text generation follow a sample-and-rerank paradigm: they draw multiple candidates, score each under a utility (reward) function using consensus across samples, and return the best one. Although effective, these methods incur high computational costs during inference due to repeated sampling and scoring. Prior attempts to amortize inference-time computation typically rely on gold references, teacher labels, or curated preference data, increasing dataset construction effort and the demand for high-fidelity reward models. We propose Consensus Group Relative Policy Optimization (C-GRPO), which distills Minimum Bayes Risk (MBR) decoding into training by formulating the consensus utility as a group-relative objective within GRPO. C-GRPO requires only a utility function and policy samples, without gold references or explicit preference labels. Under ideal conditions, we show that the objective function of C-GRPO is directionally aligned with the gradient of the expected-utility objective underlying MBR decoding, leading to a convergence guarantee. Experiments on machine translation (WMT 2024) and text summarization (XSum) demonstrate that C-GRPO successfully achieves performance comparable to MBR decoding without the associated inference-time overhead, while outperforming reference-free baseline methods.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 69 references that are in the map. First posted on arXiv 3 Feb 2026.

Builds on

46 of 69 references.

Closest ancestors · 13

Same direction, newest first

  1. 2026Silence the Judge: Reinforcement Learning with Self-Verifier via Latent Geometric Clustering13 Jan
  2. 2025Mitigating Think-Answer Mismatch in LLM Reasoning Through Noise-Aware Advantage Reweighting8 Aug
  3. 2025Geometric-Mean Policy Optimization28 Jul
  4. 2025Infinite Sampling: Efficient and Stable Grouped RL Training for Large Language Models28 Jun
  5. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  6. 2025DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO9 Jun
  7. 2025Prefix Grouper: Efficient GRPO Training through Shared-Prefix Forward5 Jun
  8. 2025Multi-Layer GRPO: Enhancing Reasoning and Self-Correction in Large Language Models5 Jun
  9. 2025Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training28 May
  10. 2025DanceGRPO: Unleashing GRPO on Visual Generation12 May
  11. 2025Flow-GRPO: Training Flow Matching Models via Online RL8 May
  12. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
1 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Model29 May

Other references · 33

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2025Qwen3 Technical Reportcited by 8,195
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  6. 2020Learning to summarize from human feedbackcited by 2,562
  7. 2021WebGPT: Browser-assisted question-answering with human feedbackcited by 1,722
  8. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  9. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  10. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  11. 2025Group Sequence Policy Optimizationcited by 788
  12. 2016Sequence-Level Knowledge Distillationcited by 761
21 more
  1. 2020Is MAP Decoding All You Need? The Inadequacy of the Mode in Neural Machine Translationcited by 137
  2. 2023From CLIP to DINO: Visual Encoders Shout in Multi-modal Large Language Modelscited by 96
  3. 2024Decoding-Time Language Model Alignment with Multiple Objectivescited by 90
  4. 2024BOND: Aligning LLMs with Best-of-N Distillationcited by 73
  5. 2021Sampling-Based Approximations to Minimum Bayes Risk Decoding for Neural Machine Translationcited by 63
  6. 2021Understanding the Properties of Minimum Bayes Risk Decoding in Neural Machine Translationcited by 56
  7. 2023It's MBR All the Way Down: Modern Generation Techniques Through the Lens of Minimum Bayes Riskcited by 49
  8. 2025MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learningcited by 48
  9. 2025M-Prometheus: A Suite of Open Multilingual LLM Judgescited by 37
  10. 2023Faster Minimum Bayes Risk Decoding with Confidence-based Pruningcited by 34
  11. 2023MBR and QE Finetuning: Training-time Distillation of the Best and Most Expensive Decoding Methodscited by 33
  12. 2023Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decodingcited by 28
  13. 2024Better Instruction-Following Through Minimum Bayes Riskcited by 19
  14. 2023An Empirical Study of Translation Hypothesis Ensembling with Large Language Modelscited by 16
  15. 2025Hybrid Group Relative Policy Optimization: A Multi-Sample Approach to Enhancing Policy Optimizationcited by 15
  16. 2024Introducing the NewsPaLM MBR and QE Dataset: LLM-Generated High-Quality Parallel Data Outperforms Traditional Web-Crawled Datacited by 10
  17. 2024Don't Throw Away Data: Better Sequence Knowledge Distillationcited by 9
  18. 2024JBBQ: Japanese Bias Benchmark for Analyzing Social Biases in Large Language Modelscited by 7
  19. 2026TranslateGemma Technical Reportcited by 7
  20. 2025Extending Group Relative Policy Optimization to Continuous Control: A Theoretical Framework for Robotic Reinforcement Learningcited by 2
  21. 2025VoiceGRPO: Modern MoE Transformers with Group Relative Policy Optimization GRPO for AI Voice Health Care Applications on Voice Pathology Detectioncited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026GRRM: Group Relative Reward Modeling for Machine Translation
  2. 2026Demystifying Group Relative Policy Optimization: Its Policy Gradient is a U-Statistic
  3. 2025GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control
  4. 2025FastGRPO: Accelerating Policy Optimization via Concurrency-aware Speculative Decoding and Online Draft Learning
  5. 2025Group Causal Policy Optimization for Post-Training Large Language Models
  6. 2026CoDistill-GRPO: A Co-Distillation Recipe for Efficient Group Relative Policy Optimization
  7. 2026Holder Policy Optimisation
  8. 2025Multi-GRPO: Multi-Group Advantage Estimation for Text-to-Image Generation with Tree-Based Trajectories and Multiple Rewards