# M3R-Bench: A Unified Benchmark for Evidence-Grounded Multimodal Metaphor Understanding

arXiv: https://arxiv.org/abs/2608.05817 · HTML: https://mlsanity.com/paper/2608.05817/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 27 references that are in the map.
- First posted on arXiv 6 Aug 2026.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency](https://mlsanity.com/paper/2508.18265/) (2025)
- [Kimi K2.5: Visual Agentic Intelligence](https://mlsanity.com/paper/2602.02276/) (2026)
- [Can MLLMs Understand the Deep Implication Behind Chinese Images?](https://mlsanity.com/paper/2410.13854/) (2024)
- [CMDAG: A Chinese Metaphor Dataset with Annotated Grounds as CoT for Boosting Metaphor Generation](https://mlsanity.com/paper/2402.13145/) (2024)
- [EmoMeta: A Multimodal Dataset for Fine-grained Emotion Classification in Chinese Metaphors](https://mlsanity.com/paper/2505.13483/) (2025)
- [Towards Multimodal Metaphor Understanding: A Chinese Dataset and Model for Metaphor Mapping Identification](https://mlsanity.com/paper/2501.02434/) (2025)

## Abstract (arXiv)

Multimodal metaphor understanding requires identifying cross-domain Target-Source mappings and inferring sentiment from visual and textual evidence. However, existing benchmarks largely evaluate these subtasks separately and lack evidence-grounded explanations, limiting their ability to assess coherent cross-modal reasoning. To enable unified and evidence-grounded evaluation, we introduce M3R-Bench, a benchmark of 1,000 human-annotated image-text instances with joint labels for metaphor occurrence, Target-Source mapping, and sentiment, alongside stage-wise explanations for evidence identification, mapping establishment, and sentiment inference. Evaluations reveal that existing models frequently rely on textual shortcuts and produce Target-Source mappings inconsistent with visual evidence. To address this evidence-mapping mismatch, we propose M3R-Reasoner, which combines curriculum-based reasoning supervision with task-aware reinforcement learning. Built on an 8B-parameter backbone, M3R-Reasoner achieves 37.90% Joint Label Accuracy (JLA) metric, outperforming the strongest proprietary baseline by 19.50 percentage points. It further surpasses GPT-5.5 in image-text evidence and sentiment justification scores by 28.45 and 28.11 points, respectively. The dataset and code are available at https://github.com/hongshi4/m3r-bench-artifact.