# From Refuse to Richness: Rubric Rewards for Long-Form Hallucination Reinforcement Learning

arXiv: https://arxiv.org/abs/2608.12337 · HTML: https://mlsanity.com/paper/2608.12337/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods.
- Builds on 12 of its 21 references that are in the map.
- First posted on arXiv 3 Jun 2026.

## Placed in

- Deep reinforcement learning methods / [RL training of language and agent models](https://mlsanity.com/deeprl/rl-training-of-language-and-agent-models/)

## Builds on (most cited first, 12 of 12)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [HybridFlow: A Flexible and Efficient RLHF Framework](https://mlsanity.com/paper/2409.19256/) (2024)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [WildChat: 1M ChatGPT Interaction Logs in the Wild](https://mlsanity.com/paper/2405.01470/) (2024)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscation](https://mlsanity.com/paper/2503.11926/) (2025)
- [Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedback](https://mlsanity.com/paper/2403.18349/) (2024)
- [Are Reasoning Models More Prone to Hallucination?](https://mlsanity.com/paper/2505.23646/) (2025)
- [CL-bench: A Benchmark for Context Learning](https://mlsanity.com/paper/2602.03587/) (2026)
- [The Hallucination Tax of Reinforcement Finetuning](https://mlsanity.com/paper/2505.13988/) (2025)
- [Learning to Reason for Factuality](https://mlsanity.com/paper/2508.05618/) (2025)
- [BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMs](https://mlsanity.com/paper/2505.13529/) (2025)

## Built on by (most cited first, 1 of 1)

- [RGDT-Bench: Benchmarking LLM Reasoning for Rule-Governed Decisions and Their Justifications](https://mlsanity.com/paper/2609.34455/) (2026)

## Abstract (arXiv)

Rewards that penalize unsupported claims can improve grounding in long-form generation, but they can also teach models to answer less. We study this refusal-to-richness trade-off in long-form hallucination RL. Instead of using global richness proxies such as length, claim count, detail, or pairwise relevance, we represent each question with a key-point rubric that specifies the required and optional information a useful answer should cover. These rubrics define coverage directly and are used both for evaluation and as reward signals. Across grounding-only, proxy-based, rubric-only, and combined rewards, we find a stable trade-off: strict grounding rewards improve support but suppress coverage, while unconstrained rubric rewards improve coverage but weaken grounding. A soft combination of grounding, rubric coverage, and relevance gives the best balance in our experiments, improving in-distribution support while transferring better to out-of-distribution checklist tasks than either grounding-only or rubric-only rewards.