# Can Large Vision-Language Models Understand Multimodal Sarcasm?

arXiv: https://arxiv.org/abs/2508.03654 · HTML: https://mlsanity.com/paper/2508.03654/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 23 of its 51 references that are in the map.
- First posted on arXiv 5 Aug 2025.

## Placed in

- Multimodal LLMs / vision-language models / [MLLM training and visual instruction tuning](https://mlsanity.com/mllm/mllm-training-and-visual-instruction-tuning/)

## Builds on (most cited first, 23 of 23)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning](https://mlsanity.com/paper/2305.06500/) (2023)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)
- [Multimodal Chain-of-Thought Reasoning in Language Models](https://mlsanity.com/paper/2302.00923/) (2023)
- [DDCoT: Duty-Distinct Chain-of-Thought Prompting for Multimodal Reasoning in Language Models](https://mlsanity.com/paper/2310.16436/) (2023)
- [GOAT-Bench: Safety Insights to Large Multimodal Models through Meme-Based Social Abuse](https://mlsanity.com/paper/2401.01523/) (2024)
- [Beyond Task Performance: Evaluating and Reducing the Flaws of Large Multimodal Models with In-Context Learning](https://mlsanity.com/paper/2310.00647/) (2023)
- [LEMMA: Towards LVLM-Enhanced Multimodal Misinformation Detection with External Knowledge Augmentation](https://mlsanity.com/paper/2402.11943/) (2024)
- [Modelling Context with User Embeddings for Sarcasm Detection in Social Media](https://mlsanity.com/paper/1607.00976/) (2016)
- [Nice perfume. How long did you marinate in it? Multimodal Sarcasm Explanation](https://mlsanity.com/paper/2112.04873/) (2021)
- [Multi-source Semantic Graph-based Multimodal Sarcasm Explanation Generation](https://mlsanity.com/paper/2306.16650/) (2023)
- [Counterfactual Reasoning for Out-of-distribution Multimodal Sentiment Analysis](https://mlsanity.com/paper/2207.11652/) (2022)
- [WisdoM: Improving Multimodal Sentiment Analysis by Fusing Contextual World Knowledge](https://mlsanity.com/paper/2401.06659/) (2024)
- [Are Word Embedding-based Features Useful for Sarcasm Detection?](https://mlsanity.com/paper/1610.00883/) (2016)
- [Stylized Data-to-Text Generation: A Case Study in the E-Commerce Domain](https://mlsanity.com/paper/2305.03256/) (2023)
- [MM-InstructEval: Zero-Shot Evaluation of (Multimodal) Large Language Models on Multimodal Reasoning Tasks](https://mlsanity.com/paper/2405.07229/) (2024)
- [General Debiasing for Multimodal Sentiment Analysis](https://mlsanity.com/paper/2307.10511/) (2023)
- [Fine-grained and Explainable Factuality Evaluation for Multimodal Summarization](https://mlsanity.com/paper/2402.11414/) (2024)
- [CofiPara: A Coarse-to-fine Paradigm for Multimodal Sarcasm Target Identification with Large Multimodal Models](https://mlsanity.com/paper/2405.00390/) (2024)
- [Sentiment-enhanced Graph-based Sarcasm Explanation in Dialogue](https://mlsanity.com/paper/2402.03658/) (2024)
- [Rainbow Noise: Stress-Testing Multimodal Harmful-Meme Detectors on LGBTQ Content](https://mlsanity.com/paper/2507.19551/) (2025)
- [Defeasible Visual Entailment: Benchmark, Evaluator, and Reward-Driven Optimization](https://mlsanity.com/paper/2412.16232/) (2024)

## Built on by (most cited first, 5 of 5)

- ["I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?](https://mlsanity.com/paper/2604.05930/) (2026)
- [Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding](https://mlsanity.com/paper/2601.17197/) (2026)
- [GRASP: Grounded CoT Reasoning with Dual-Stage Optimization for Multimodal Sarcasm Target Identification](https://mlsanity.com/paper/2604.08879/) (2026)
- [MetaphorVU: Towards Metaphorical Video Understanding](https://mlsanity.com/paper/2605.25461/) (2026)
- [Beyond Visual Similarity: Entity-Aligned Retrieval for Knowledge-Based Visual Question Answering](https://mlsanity.com/paper/2608.21450/) (2026)

## Abstract (arXiv)

Sarcasm is a complex linguistic phenomenon that involves a disparity between literal and intended meanings, making it challenging for sentiment analysis and other emotion-sensitive tasks. While traditional sarcasm detection methods primarily focus on text, recent approaches have incorporated multimodal information. However, the application of Large Visual Language Models (LVLMs) in Multimodal Sarcasm Analysis (MSA) remains underexplored. In this paper, we evaluate LVLMs in MSA tasks, specifically focusing on Multimodal Sarcasm Detection and Multimodal Sarcasm Explanation. Through comprehensive experiments, we identify key limitations, such as insufficient visual understanding and a lack of conceptual knowledge. To address these issues, we propose a training-free framework that integrates in-depth object extraction and external conceptual knowledge to improve the model's ability to interpret and explain sarcasm in multimodal contexts. The experimental results on multiple models show the effectiveness of our proposed framework. The code is available at https://github.com/cp-cp/LVLM-MSA.