# EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning

arXiv: https://arxiv.org/abs/2610.13353 · HTML: https://mlsanity.com/paper/2610.13353/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 23 of its 25 references that are in the map.
- First posted on arXiv 9 Oct 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Reward modeling and RLHF](https://mlsanity.com/alignment/reward-modeling-and-rlhf/)

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Qwen2.5-Omni Technical Report](https://mlsanity.com/paper/2503.20215/) (2025)
- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning](https://mlsanity.com/paper/2406.11161/) (2024)
- [Explainable Multimodal Emotion Recognition](https://mlsanity.com/paper/2306.15401/) (2023)
- [EmotionThinker: Prosody-Aware Reinforcement Learning for Explainable Speech Emotion Reasoning](https://mlsanity.com/paper/2601.15668/) (2026)
- [When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Models](https://mlsanity.com/paper/2508.15407/) (2025)
- [EmotionHallucer: Evaluating Emotion Hallucinations in Multimodal Large Language Models](https://mlsanity.com/paper/2505.11405/) (2025)
- [EmoPrefer: Can Large Language Models Understand Human Emotion Preferences?](https://mlsanity.com/paper/2507.04278/) (2025)
- [MER 2026: From Discriminative Emotion Recognition to Generative Emotion Understanding](https://mlsanity.com/paper/2604.19417/) (2026)
- [AVERE: Improving Audiovisual Emotion Reasoning with Preference Optimization](https://mlsanity.com/paper/2602.07054/) (2026)
- [Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models](https://mlsanity.com/paper/2606.05161/) (2026)
- [EII-SCL: Harnessing Emotional Inertia for Multimodal Emotion Recognition in Conversation](https://mlsanity.com/paper/2607.17366/) (2026)
- [EmoEUS: Uncertainty Supervision for Multimodal Emotion Recognition in Conversation](https://mlsanity.com/paper/2607.18336/) (2026)
- [Learning to Prefer Reliably: Error-Augmented Emotion Preference Optimization with Calibrated Fusion](https://mlsanity.com/paper/2608.24730/) (2026)

## Abstract (arXiv)

Multimodal emotion preference learning uses pairwise comparisons to identify which open-ended emotion descriptions align more closely with human judgments. However, existing methods primarily focus on the final preference decision and provide little explicit supervision for localizing and correcting erroneous content. To address this limitation, we propose EmoGavel, a Grounding-Aware reinforcement learning framework with Verifiable rewards for multimodal Emotion preference Learning. We first introduce grounded supervised fine-tuning (Grounded SFT), which exploits controlled-error construction information to jointly learn preference prediction, error localization, repair, and missing-evidence recovery. We then optimize these verifiable outcomes with grounding-aware group relative policy optimization (GA-GRPO), combining preference-gated grounding rewards with dynamic zero-variance rescue to prioritize preference correctness and recover learning signals from all-wrong groups. Experiments on EmoPrefer and controlled-error evaluation sets show that EmoGavel improves both natural preference prediction and controlled-error discrimination while maintaining strong error-grounding performance. Our code and model checkpoints will be released once the paper is accepted.