# Learning to Prefer Reliably: Error-Augmented Emotion Preference Optimization with Calibrated Fusion

arXiv: https://arxiv.org/abs/2608.24730 · HTML: https://mlsanity.com/paper/2608.24730/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal emotion understanding, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 13 of its 31 references that are in the map.
- First posted on arXiv 25 Aug 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Multimodal emotion understanding](https://mlsanity.com/mllm/multimodal-emotion-understanding/)
- LLM alignment / RLHF / preference optimisation / [Medical LLM alignment](https://mlsanity.com/alignment/medical-llm-alignment/)

## Builds on (most cited first, 13 of 13)

- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [Emotion-LLaMA: Multimodal Emotion Recognition and Reasoning with Instruction Tuning](https://mlsanity.com/paper/2406.11161/) (2024)
- [When Audio and Text Disagree: Revealing Text Bias in Large Audio-Language Models](https://mlsanity.com/paper/2508.15407/) (2025)
- [EmoPrefer: Can Large Language Models Understand Human Emotion Preferences?](https://mlsanity.com/paper/2507.04278/) (2025)
- [RBA-GCN: Relational Bilevel Aggregation Graph Convolutional Network for Emotion Recognition](https://mlsanity.com/paper/2308.11029/) (2023)
- [Enhancing Multimodal Emotion Recognition through Multi-Granularity Cross-Modal Alignment](https://mlsanity.com/paper/2412.20821/) (2024)
- [MER 2026: From Discriminative Emotion Recognition to Generative Emotion Understanding](https://mlsanity.com/paper/2604.19417/) (2026)
- [Beyond Text Following: Repairable Arbitration Reversals in Audio-Language Models](https://mlsanity.com/paper/2606.05161/) (2026)
- [EII-SCL: Harnessing Emotional Inertia for Multimodal Emotion Recognition in Conversation](https://mlsanity.com/paper/2607.17366/) (2026)

## Built on by (most cited first, 1 of 1)

- [EmoGavel: Grounding-Aware Reinforcement Learning with Verifiable Rewards for Multimodal Emotion Preference Learning](https://mlsanity.com/paper/2610.13353/) (2026)

## Abstract (arXiv)

Emotion preference learning uses pairwise comparisons between candidate descriptions to align multimodal large language models (MLLMs) with human judgments of open-ended emotion descriptions and to train reward models that capture human emotional preferences. However, conventional pairwise supervision is often sparse, typically providing only a single negative description for each positive description, and therefore offers limited coverage of the diverse ways in which an emotion description can be incorrect. In particular, models may be insufficiently exposed to semantically fluent but emotionally inconsistent descriptions. Beyond this data-level limitation, relying on a single MLLM judge introduces a distinct model-level concern: its judgments can be affected by model-specific biases when interpreting fine-grained or ambiguous multimodal emotional cues. To address these limitations, we propose Error-Augmented Preference Optimization (EAPO), a framework for improving the reliability of MLLM-based emotion preference judgment at both the data and model levels. First, we construct an error-augmented dataset by generating multiple controlled and emotion-aware negative descriptions from each preferred description. We then adapt multiple independent MLLM judges to this richer supervision and aggregate their preference margins using margin-calibrated soft fusion, which maps heterogeneous margins to a common scale before aggregation. Experiments on the MER2026-EmoPrefer Challenge dataset and our error-augmented dataset demonstrate that EAPO improves emotion preference prediction and enhances the robustness of MLLM judges when evaluating fluent descriptions that conflict with the video's multimodal emotional evidence. Our code is available at https://github.com/slash1028/EAPO-EmoPrefer.