mlsanityData to 7 Oct 2026

"I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?

arXiv 2604.05930 ↗First posted 7 Apr 2026cs.CL cs.AIcited by 1

On the map: Multimodal LLMs / vision-language models → MLLM training and visual instruction tuning

Abstract (arXiv)

Puns are a common form of rhetorical wordplay that exploits polysemy and phonetic similarity to create humor. In multimodal puns, visual and textual elements synergize to ground the literal sense and evoke the figurative meaning simultaneously. Although Vision-Language Models (VLMs) are widely used in multimodal understanding and generation, their ability to understand puns has not been systematically studied due to a scarcity of rigorous benchmarks. To address this, we first propose a multimodal pun generation pipeline. We then introduce MultiPun, a dataset comprising diverse types of puns alongside adversarial non-pun distractors. Our evaluation reveals that most models struggle to distinguish genuine puns from these distractors. Moreover, we propose both prompt-level and model-level strategies to enhance pun comprehension, with an average improvement of 16.5% in F1 scores. Our findings provide valuable insights for developing future VLMs that master the subtleties of human-like humor via cross-modal reasoning.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in MLLM training and visual instruction tuning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 50 references that are in the map. First posted on arXiv 7 Apr 2026.

Builds on

21 of 50 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Can Large Vision-Language Models Understand Multimodal Sarcasm?5 Aug

Other references · 20

  1. 2023Beyond Yes and No: Improving Zero-Shot LLM Rankers via Scoring Fine-Grained Relevance Labelscited by 86
  2. 2025IPIGuard: A Novel Tool Dependency Graph-Based Defense Against Indirect Prompt Injection in LLM Agentscited by 46
  3. 2022ExPUNations: Augmenting Puns with Keywords and Explanationscited by 21
  4. 2024"A good pun is its own reword": Can Large Language Models Understand Puns?cited by 21
  5. 2025MaPPO: Maximum a Posteriori Preference Optimization with Prior Knowledgecited by 14
  6. 2024Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictionscited by 11
  7. 2024Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understandingcited by 11
  8. 2025CTCC: A Robust and Stealthy Fingerprinting Framework for Large Language Models via Cross-Turn Contextual Correlation Backdoorcited by 11
  9. 2024Breaking the Memory Wall for Heterogeneous Federated Learning via Progressive Trainingcited by 10
  10. 2024Heterogeneity-Aware Memory Efficient Federated Learning via Progressive Layer Freezingcited by 10
  11. 2019Joint Detection and Location of English Punscited by 9
  12. 2020"The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognitioncited by 9
8 more
  1. 2024Fingerprint Vector: Enabling Scalable and Efficient Model Fingerprint Transfer via Vector Additioncited by 9
  2. 2025Bridging the Copyright Gap: Do Large Vision-Language Models Recognize and Respect Copyrighted Content?cited by 8
  3. 2026AdaMARP: An Adaptive Multi-Agent Interaction Framework for General Immersive Role-Playingcited by 8
  4. 2024Can visual language models resolve textual ambiguity with visual cues? Let visual puns tell you!cited by 7
  5. 2026TSEmbed: Unlocking Task Scaling in Universal Multimodal Embeddingscited by 6
  6. 2025Memory-Efficient Federated Fine-Tuning of Large Language Models via Layer Pruningcited by 5
  7. 2025Elastic Mixture of Rank-Wise Experts for Knowledge Reuse in Federated Fine-Tuningcited by 5
  8. 2025Pun Unintended: LLMs and the Illusion of Humor Understandingcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Spotlight and Shadow: Attention-Guided Dual-Anchor Introspective Decoding for MLLM Hallucination Mitigationcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2026Reasoning Beyond Literal: Cross-style Multimodal Reasoning for Figurative Language Understanding
  2. 2025Visual Puns from Idioms: An Iterative LLM-T2IM-MLLM Framework
  3. 2025Seeing Sarcasm Through Different Eyes: Analyzing Multimodal Sarcasm Perception in Large Vision-Language Models
  4. 2026Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges
  5. 2024POEM: Interactive Prompt Optimization for Enhancing Multimodal Reasoning of Large Language Models
  6. 2024Understanding Figurative Meaning through Explainable Visual Entailment
  7. 2025Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings for CLEF JOKER 2025 Task 2
  8. 2026CaRGo-T: Causal Reasoning Graph-of-Thought improves Multimodal Humor Comprehension