# "A good pun is its own reword": Can Large Language Models Understand Puns?

arXiv: https://arxiv.org/abs/2404.13599 · HTML: https://mlsanity.com/paper/2404.13599/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 45 references that are in the map.
- First posted on arXiv 21 Apr 2024.

## Builds on (most cited first, 21 of 21)

- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [OpenChat: Advancing Open-source Language Models with Mixed-Quality Data](https://mlsanity.com/paper/2309.11235/) (2023)
- [EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instruction](https://mlsanity.com/paper/2401.06201/) (2024)
- [Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest](https://mlsanity.com/paper/2209.06293/) (2022)
- [Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmark](https://mlsanity.com/paper/2305.14938/) (2023)
- [Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generation](https://mlsanity.com/paper/2312.02439/) (2023)
- [Pun Generation with Surprise](https://mlsanity.com/paper/1904.06828/) (2019)
- [Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Models](https://mlsanity.com/paper/2308.13961/) (2023)
- [AmbiPun: Generating Humorous Puns with Ambiguous Context](https://mlsanity.com/paper/2205.01825/) (2022)
- [ExPUNations: Augmenting Puns with Keywords and Explanations](https://mlsanity.com/paper/2210.13513/) (2022)
- [Small But Funny: A Feedback-Driven Approach to Humor Distillation](https://mlsanity.com/paper/2402.18113/) (2024)
- [Can Language Models Laugh at YouTube Short-form Videos?](https://mlsanity.com/paper/2310.14159/) (2023)
- [A Unified Framework for Pun Generation with Humor Principles](https://mlsanity.com/paper/2210.13055/) (2022)
- [Joint Detection and Location of English Puns](https://mlsanity.com/paper/1909.00175/) (2019)
- ["The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognition](https://mlsanity.com/paper/2004.14457/) (2020)
- [Witscript 3: A Hybrid AI System for Improvising Jokes in a Conversation](https://mlsanity.com/paper/2301.02695/) (2023)
- [Humor@IITK at SemEval-2021 Task 7: Large Language Models for Quantifying Humor and Offensiveness](https://mlsanity.com/paper/2104.00933/) (2021)

## Built on by (most cited first, 21 of 21)

- [Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning](https://mlsanity.com/paper/2406.10522/) (2024)
- [CLEAR: Can Language Models Really Understand Causal Graphs?](https://mlsanity.com/paper/2406.16605/) (2024)
- [Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leaps](https://mlsanity.com/paper/2410.10370/) (2024)
- [Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understanding](https://mlsanity.com/paper/2406.10318/) (2024)
- [A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Models](https://mlsanity.com/paper/2501.15147/) (2025)
- [Text Is Not All You Need: Multimodal Prompting Helps LLMs Understand Humor](https://mlsanity.com/paper/2412.05315/) (2024)
- [STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasings](https://mlsanity.com/paper/2504.13416/) (2025)
- [GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Art](https://mlsanity.com/paper/2505.11436/) (2025)
- [Who's Laughing Now? An Overview of Computational Humour Generation and Explanation](https://mlsanity.com/paper/2509.21175/) (2025)
- [Pun Unintended: LLMs and the Illusion of Humor Understanding](https://mlsanity.com/paper/2509.12158/) (2025)
- [A Survey of Pun Generation: Datasets, Evaluations and Methodologies](https://mlsanity.com/paper/2507.04793/) (2025)
- [Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworks](https://mlsanity.com/paper/2507.11292/) (2025)
- ["I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?](https://mlsanity.com/paper/2604.05930/) (2026)
- [Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings for CLEF JOKER 2025 Task 2](https://mlsanity.com/paper/2507.06506/) (2025)
- [Words at Play: Benchmarking Audio Pun Understanding in Large Audio-Language Models](https://mlsanity.com/paper/2603.18678/) (2026)
- [KoWit-24: A Richly Annotated Dataset of Wordplay in News Headlines](https://mlsanity.com/paper/2503.01510/) (2025)
- [One Joke to Rule them All? On the (Im)possibility of Generalizing Humor](https://mlsanity.com/paper/2508.19402/) (2025)
- [Phun-Bench: Evaluating LLMs on Phonological Understanding in Chinese](https://mlsanity.com/paper/2606.07300/) (2026)
- [Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?](https://mlsanity.com/paper/2607.23440/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)
- [Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings](https://mlsanity.com/paper/2608.04311/) (2026)

## Abstract (arXiv)

Puns play a vital role in academic research due to their distinct structure and clear definition, which aid in the comprehensive analysis of linguistic humor. However, the understanding of puns in large language models (LLMs) has not been thoroughly examined, limiting their use in creative writing and humor creation. In this paper, we leverage three popular tasks, i.e., pun recognition, explanation and generation to systematically evaluate the capabilities of LLMs in pun understanding. In addition to adopting the automated evaluation metrics from prior research, we introduce new evaluation methods and metrics that are better suited to the in-context learning paradigm of LLMs. These new metrics offer a more rigorous assessment of an LLM's ability to understand puns and align more closely with human cognition than previous metrics. Our findings reveal the "lazy pun generation" pattern and identify the primary challenges LLMs encounter in understanding puns.