# Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding

arXiv: https://arxiv.org/abs/2601.17197 · HTML: https://mlsanity.com/paper/2601.17197/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Multimodal reasoning, one of 17 directions in LLM reasoning.
- Builds on 23 of its 40 references that are in the map.
- First posted on arXiv 23 Jan 2026.

## Placed in

- LLM reasoning / [Multimodal reasoning](https://mlsanity.com/lr/multimodal-reasoning/)
- Multimodal LLMs / vision-language models / [VLM benchmarks and visual reasoning](https://mlsanity.com/mllm/vlm-benchmarks-and-visual-reasoning/)

## Builds on (most cited first, 23 of 23)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts](https://mlsanity.com/paper/2310.02255/) (2023)
- [Least-to-Most Prompting Enables Complex Reasoning in Large Language Models](https://mlsanity.com/paper/2205.10625/) (2022)
- [Decomposed Prompting: A Modular Approach for Solving Complex Tasks](https://mlsanity.com/paper/2210.02406/) (2022)
- [VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model](https://mlsanity.com/paper/2504.07615/) (2025)
- [Large Language Models Are Reasoning Teachers](https://mlsanity.com/paper/2212.10071/) (2022)
- [Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs](https://mlsanity.com/paper/2506.14245/) (2025)
- [Improve Vision Language Model Chain-of-thought Reasoning](https://mlsanity.com/paper/2410.16198/) (2024)
- [Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning](https://mlsanity.com/paper/2505.14677/) (2025)
- [FLUTE: Figurative Language Understanding through Textual Explanations](https://mlsanity.com/paper/2205.12404/) (2022)
- [Does BERT Learn as Humans Perceive? Understanding Linguistic Styles through Lexica](https://mlsanity.com/paper/2109.02738/) (2021)
- [IRFL: Image Recognition of Figurative Language](https://mlsanity.com/paper/2303.15445/) (2023)
- [STAR-R1: Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs](https://mlsanity.com/paper/2505.15804/) (2025)
- [SVQA-R1: Reinforcing Spatial Reasoning in MLLMs via View-Consistent Reward Optimization](https://mlsanity.com/paper/2506.01371/) (2025)
- [RePIC: Reinforced Post-Training for Personalizing Multi-Modal Language Models](https://mlsanity.com/paper/2506.18369/) (2025)
- [Can Large Vision-Language Models Understand Multimodal Sarcasm?](https://mlsanity.com/paper/2508.03654/) (2025)
- [Style is NOT a single variable: Case Studies for Cross-Style Language Understanding](https://mlsanity.com/paper/1911.03663/) (2019)
- [Multilingual Multi-Figurative Language Detection](https://mlsanity.com/paper/2306.00121/) (2023)
- [The MuSe 2024 Multimodal Sentiment Analysis Challenge: Social Perception and Humor Recognition](https://mlsanity.com/paper/2406.07753/) (2024)
- [A Comparative Study on Textual Saliency of Styles from Eye Tracking, Annotations, and Language Models](https://mlsanity.com/paper/2212.09873/) (2022)
- [Balancing Effect of Training Dataset Distribution of Multiple Styles for Multi-Style Text Transfer](https://mlsanity.com/paper/2305.15582/) (2023)

## Abstract (arXiv)

Vision-language models (VLMs) have demonstrated strong reasoning abilities in literal multimodal tasks such as visual mathematics and science question answering. However, figurative language, such as sarcasm, humor, and metaphor, remains a significant challenge, as it conveys intent and emotion through subtle incongruities between expressed and intended meanings. In multimodal settings, accompanying images can amplify or invert textual meaning, demanding models that reason across modalities and account for subjectivity. We propose a three-step framework for developing efficient multimodal reasoning models that can (i) interpret multimodal figurative language, (ii) provide transparent reasoning traces, and (iii) generalize across multiple figurative styles. Experiments across four styles show that (1) incorporating reasoning traces substantially improves multimodal figurative understanding, (2) reasoning learned in one style can transfer to others, especially between related styles like sarcasm and humor, and (3) training jointly across styles yields a generalized reasoning VLM that outperforms much larger open- and closed-source models. Our findings show that lightweight VLMs with verifiable reasoning achieve robust cross-style generalization while providing inspectable reasoning traces for multimodal tasks. The code and implementation are available at https://github.com/scheshmi/CrossStyle-MMR.