# Pun Generation with Surprise

arXiv: https://arxiv.org/abs/1904.06828 · HTML: https://mlsanity.com/paper/1904.06828/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Apr 2019.

## Built on by (most cited first, 30 of 37)

- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://mlsanity.com/paper/2210.15097/) (2022)
- [The Perils of Using Mechanical Turk to Evaluate Open-Ended Text Generation](https://mlsanity.com/paper/2109.06835/) (2021)
- [RiddleSense: Reasoning about Riddle Questions Featuring Linguistic Creativity and Commonsense Knowledge](https://mlsanity.com/paper/2101.00376/) (2021)
- [HypoGen: Hyperbole Generation with Commonsense and Counterfactual Knowledge](https://mlsanity.com/paper/2109.05097/) (2021)
- [AmbiPun: Generating Humorous Puns with Ambiguous Context](https://mlsanity.com/paper/2205.01825/) (2022)
- [ExPUNations: Augmenting Puns with Keywords and Explanations](https://mlsanity.com/paper/2210.13513/) (2022)
- ["A good pun is its own reword": Can Large Language Models Understand Puns?](https://mlsanity.com/paper/2404.13599/) (2024)
- [Decrypting Cryptic Crosswords: Semantically Complex Wordplay Puzzles as a Target for NLP](https://mlsanity.com/paper/2104.08620/) (2021)
- [TwistList: Resources and Baselines for Tongue Twister Generation](https://mlsanity.com/paper/2306.03457/) (2023)
- [InsNet: An Efficient, Flexible, and Performant Insertion-based Text Generation Model](https://mlsanity.com/paper/2102.11008/) (2021)
- [MOVER: Mask, Over-generate and Rank for Hyperbole Generation](https://mlsanity.com/paper/2109.07726/) (2021)
- [Can Pre-trained Language Models Understand Chinese Humor?](https://mlsanity.com/paper/2407.04105/) (2024)
- [The Iron(ic) Melting Pot: Reviewing Human Evaluation in Humour, Irony and Sarcasm Generation](https://mlsanity.com/paper/2311.05552/) (2023)
- [A Unified Framework for Pun Generation with Humor Principles](https://mlsanity.com/paper/2210.13055/) (2022)
- [OxfordTVG-HIC: Can Machine Make Humorous Captions from Images?](https://mlsanity.com/paper/2307.11636/) (2023)
- [Extrapolative Controlled Sequence Generation via Iterative Refinement](https://mlsanity.com/paper/2303.04562/) (2023)
- [Are LLMs Good Cryptic Crossword Solvers?](https://mlsanity.com/paper/2403.12094/) (2024)
- [Creativity in AI: Progresses and Challenges](https://mlsanity.com/paper/2410.17218/) (2024)
- [Can Language Models Make Fun? A Case Study in Chinese Comical Crosstalk](https://mlsanity.com/paper/2207.00735/) (2022)
- [Controllable Text Generation for Open-Domain Creativity and Fairness](https://mlsanity.com/paper/2209.12099/) (2022)
- [Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End](https://mlsanity.com/paper/2212.10522/) (2022)
- [A Two-Model Approach for Humour Style Recognition](https://mlsanity.com/paper/2410.12842/) (2024)
- [Comparing Apples to Oranges: A Dataset & Analysis of LLM Humour Understanding from Traditional Puns to Topical Jokes](https://mlsanity.com/paper/2507.13335/) (2025)
- [Context-Situated Pun Generation](https://mlsanity.com/paper/2210.13522/) (2022)
- [Prompt to GPT-3: Step-by-Step Thinking Instructions for Humor Generation](https://mlsanity.com/paper/2306.13195/) (2023)
- [Who's Laughing Now? An Overview of Computational Humour Generation and Explanation](https://mlsanity.com/paper/2509.21175/) (2025)
- [Train & Constrain: Phonologically Informed Tongue-Twister Generation from Topics and Paraphrases](https://mlsanity.com/paper/2403.13901/) (2024)
- [When a Computer Cracks a Joke: Automated Generation of Humorous Headlines](https://mlsanity.com/paper/2109.08702/) (2021)
- [IG Captioner: Information Gain Captioners are Strong Zero-shot Classifiers](https://mlsanity.com/paper/2311.17072/) (2023)
- [A Survey of Pun Generation: Datasets, Evaluations and Methodologies](https://mlsanity.com/paper/2507.04793/) (2025)

## Abstract (arXiv)

We tackle the problem of generating a pun sentence given a pair of homophones (e.g., "died" and "dyed"). Supervised text generation is inappropriate due to the lack of a large corpus of puns, and even if such a corpus existed, mimicry is at odds with generating novel content. In this paper, we propose an unsupervised approach to pun generation using a corpus of unhumorous text and what we call the local-global surprisal principle: we posit that in a pun sentence, there is a strong association between the pun word (e.g., "dyed") and the distant context, as well as a strong association between the alternative word (e.g., "died") and the immediate context. This contrast creates surprise and thus humor. We instantiate this principle for pun generation in two ways: (i) as a measure based on the ratio of probabilities under a language model, and (ii) a retrieve-and-edit approach based on words suggested by a skip-gram model. Human evaluation shows that our retrieve-and-edit approach generates puns successfully 31% of the time, tripling the success rate of a neural generation baseline.