mlsanityData to 7 Oct 2026

"A good pun is its own reword": Can Large Language Models Understand Puns?

arXiv 2404.13599 ↗First posted 21 Apr 2024cs.CLcited by 21

Not on the map. Why?

Abstract (arXiv)

Puns play a vital role in academic research due to their distinct structure and clear definition, which aid in the comprehensive analysis of linguistic humor. However, the understanding of puns in large language models (LLMs) has not been thoroughly examined, limiting their use in creative writing and humor creation. In this paper, we leverage three popular tasks, i.e., pun recognition, explanation and generation to systematically evaluate the capabilities of LLMs in pun understanding. In addition to adopting the automated evaluation metrics from prior research, we introduce new evaluation methods and metrics that are better suited to the in-context learning paradigm of LLMs. These new metrics offer a more rigorous assessment of an LLM's ability to understand puns and align more closely with human cognition than previous metrics. Our findings reveal the "lazy pun generation" pattern and identify the primary challenges LLMs encounter in understanding puns.

Key facts. Cited by 21 papers in the map (data through 7 Oct 2026). Builds on 21 of its 45 references that are in the map. First posted on arXiv 21 Apr 2024.

Builds on

21 of 45 references.

References · 21

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  3. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
  4. 2016Pointer Sentinel Mixture Modelscited by 2,992
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
  6. 2023OpenChat: Advancing Open-source Language Models with Mixed-Quality Datacited by 250
  7. 2024EASYTOOL: Enhancing LLM-based Agents with Concise Tool Instructioncited by 153
  8. 2022Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contestcited by 98
  9. 2023Do LLMs Understand Social Knowledge? Evaluating the Sociability of Large Language Models with SocKET Benchmarkcited by 62
  10. 2023Let's Think Outside the Box: Exploring Leap-of-Thought in Large Language Models with Creative Humor Generationcited by 56
  11. 2019Pun Generation with Surprisecited by 37
  12. 2023Translate Meanings, Not Just Words: IdiomKB's Role in Optimizing Idiomatic Translation with Language Modelscited by 23
9 more
  1. 2022AmbiPun: Generating Humorous Puns with Ambiguous Contextcited by 21
  2. 2022ExPUNations: Augmenting Puns with Keywords and Explanationscited by 21
  3. 2024Small But Funny: A Feedback-Driven Approach to Humor Distillationcited by 12
  4. 2023Can Language Models Laugh at YouTube Short-form Videos?cited by 11
  5. 2022A Unified Framework for Pun Generation with Humor Principlescited by 10
  6. 2019Joint Detection and Location of English Punscited by 9
  7. 2020"The Boating Store Had Its Best Sail Ever": Pronunciation-attentive Contextualized Pun Recognitioncited by 9
  8. 2023Witscript 3: A Hybrid AI System for Improvising Jokes in a Conversationcited by 8
  9. 2021Humor@IITK at SemEval-2021 Task 7: Large Language Models for Quantifying Humor and Offensivenesscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 10; 2026: 6.

21 papers.

Sort:

By area:

  1. 2024Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioningcited by 18
  2. 2024CLEAR: Can Language Models Really Understand Causal Graphs?cited by 14
  3. 2024Innovative Thinking, Infinite Humor: Humor Research of Large Language Models through Structured Thought Leapscited by 12
  4. 2024Creating a Lens of Chinese Culture: A Multimodal Dataset for Chinese Pun Rebus Art Understandingcited by 11
  5. 2025A Causality-aware Paradigm for Evaluating Creativity of Multimodal Large Language Modelscited by 10
  6. 2024Text Is Not All You Need: Multimodal Prompting Helps LLMs Understand Humorcited by 8
  7. 2025STAMP Your Content: Proving Dataset Membership via Watermarked Rephrasingscited by 8
  8. 2025GODBench: A Benchmark for Multimodal Large Language Models in Video Comment Artcited by 5
  9. 2025Who's Laughing Now? An Overview of Computational Humour Generation and Explanationcited by 5
  10. 2025A Survey of Pun Generation: Datasets, Evaluations and Methodologiescited by 2
  11. 2025Fine-Grained Chinese Hate Speech Understanding: Span-Level Resources, Coded Term Lexicon, and Enhanced Detection Frameworkscited by 2
  12. 2025Pun Unintended: LLMs and the Illusion of Humor Understandingcited by 2
9 more
  1. 2025Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddings for CLEF JOKER 2025 Task 2cited by 1
  2. 2026Words at Play: Benchmarking Audio Pun Understanding in Large Audio-Language Modelscited by 1
  3. 2026"I See What You Did There": Can Large Vision-Language Models Understand Multimodal Puns?cited by 1
  4. 2025KoWit-24: A Richly Annotated Dataset of Wordplay in News Headlinescited by 0
  5. 2025One Joke to Rule them All? On the (Im)possibility of Generalizing Humorcited by 0
  6. 2026Phun-Bench: Evaluating LLMs on Phonological Understanding in Chinesecited by 0
  7. 2026Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?cited by 0
  8. 2026Towards High-Level Semantic Intelligencecited by 0
  9. 2026Pun Intended: Multi-Agent Translation of Wordplay with Contrastive Learning and Phonetic-Semantic Embeddingscited by 0