mlsanityData to 14 Oct 2026

From Refuse to Richness: Rubric Rewards for Long-Form Hallucination Reinforcement Learning

arXiv 2608.12337 ↗First posted 3 Jun 2026cs.CLcited by 1

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Rewards that penalize unsupported claims can improve grounding in long-form generation, but they can also teach models to answer less. We study this refusal-to-richness trade-off in long-form hallucination RL. Instead of using global richness proxies such as length, claim count, detail, or pairwise relevance, we represent each question with a key-point rubric that specifies the required and optional information a useful answer should cover. These rubrics define coverage directly and are used both for evaluation and as reward signals. Across grounding-only, proxy-based, rubric-only, and combined rewards, we find a stable trade-off: strict grounding rewards improve support but suppress coverage, while unconstrained rubric rewards improve coverage but weaken grounding. A soft combination of grounding, rubric coverage, and relevance gives the best balance in our experiments, improving in-distribution support while transferring better to out-of-distribution checklist tasks than either grounding-only or rubric-only rewards.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 12 of its 21 references that are in the map. First posted on arXiv 3 Jun 2026.

Builds on

12 of 21 references.

References · 12

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  2. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  3. 2023FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generationcited by 1,282
  4. 2024WildChat: 1M ChatGPT Interaction Logs in the Wildcited by 629
  5. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  6. 2025Monitoring Reasoning Models for Misbehavior and the Risks of Promoting Obfuscationcited by 384
  7. 2024Rejection Improves Reliability: Training LLMs to Refuse Unknown Questions Using RL from Knowledge Feedbackcited by 66
  8. 2025Are Reasoning Models More Prone to Hallucination?cited by 64
  9. 2026CL-bench: A Benchmark for Context Learningcited by 47
  10. 2025The Hallucination Tax of Reinforcement Finetuningcited by 40
  11. 2025Learning to Reason for Factualitycited by 24
  12. 2025BARREL: Boundary-Aware Reasoning for Factual and Reliable LRMscited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026RGDT-Bench: Benchmarking LLM Reasoning for Rule-Governed Decisions and Their Justificationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Train for Truth, Keep the Skills: Binary Retrieval-Augmented Reward Mitigates Hallucinations
  2. 2026BALTO: Balanced Token-Level Policy Optimization for Hallucination Mitigation
  3. 2025Enhancing Reliability across Short and Long-Form QA via Reinforcement Learning
  4. 2025Learning to Reason for Hallucination Span Detection
  5. 2026Features as Rewards: Scalable Supervision for Open-Ended Tasks via Interpretability
  6. 2025QueryBandits for Hallucination Mitigation: Exploiting Semantic Features for No-Regret Rewriting
  7. 2026Stop Rewarding Hallucinated Steps: Faithfulness-Aware Step-Level Reinforcement Learning for Small Reasoning Models
  8. 2025TruthRL: Incentivizing Truthful LLMs via Reinforcement Learning