# Laughing Heads: Can Transformers Detect What Makes a Sentence Funny?

arXiv: https://arxiv.org/abs/2105.09142 · HTML: https://mlsanity.com/paper/2105.09142/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 47 references that are in the map.
- First posted on arXiv 19 May 2021.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [What Does BERT Look At? An Analysis of BERT's Attention](https://mlsanity.com/paper/1906.04341/) (2019)
- [Attention is not Explanation](https://mlsanity.com/paper/1902.10186/) (2019)
- [Attention is not not Explanation](https://mlsanity.com/paper/1908.04626/) (2019)
- [Revealing the Dark Secrets of BERT](https://mlsanity.com/paper/1908.08593/) (2019)
- [Learning Word Vectors for 157 Languages](https://mlsanity.com/paper/1802.06893/) (2018)
- [Humor Detection: A Transformer Gets the Last Laugh](https://mlsanity.com/paper/1909.00252/) (2019)
- ["President Vows to Cut <Taxes> Hair": Dataset and Analysis of Creative Text Editing for Humorous Headlines](https://mlsanity.com/paper/1906.00274/) (2019)
- [Reverse-Engineering Satire, or "Paper on Computational Humor Accepted Despite Making Serious Advances"](https://mlsanity.com/paper/1901.03253/) (2019)
- [Stimulating Creativity with FunLines: A Case Study of Humor Generation in Headlines](https://mlsanity.com/paper/2002.02031/) (2020)

## Built on by (most cited first, 9 of 9)

- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [A Glitch in the Matrix? Locating and Detecting Language Model Grounding with Fakepedia](https://mlsanity.com/paper/2312.02073/) (2023)
- [Getting Serious about Humor: Crafting Humor Datasets with Unfunny Large Language Models](https://mlsanity.com/paper/2403.00794/) (2024)
- [Vision Transformers: State of the Art and Research Challenges](https://mlsanity.com/paper/2207.03041/) (2022)
- [Representation biases in sentence transformers](https://mlsanity.com/paper/2301.13039/) (2023)
- [Creativity in AI: Progresses and Challenges](https://mlsanity.com/paper/2410.17218/) (2024)
- [Transformers Go for the LOLs: Generating (Humourous) Titles from Scientific Abstracts End-to-End](https://mlsanity.com/paper/2212.10522/) (2022)
- [The Naughtyformer: A Transformer Understands Offensive Humor](https://mlsanity.com/paper/2211.14369/) (2022)
- [Toward Machine Interpreting: Lessons from Human Interpreting Studies](https://mlsanity.com/paper/2508.07964/) (2025)

## Abstract (arXiv)

The automatic detection of humor poses a grand challenge for natural language processing. Transformer-based systems have recently achieved remarkable results on this task, but they usually (1)~were evaluated in setups where serious vs humorous texts came from entirely different sources, and (2)~focused on benchmarking performance without providing insights into how the models work. We make progress in both respects by training and analyzing transformer-based humor recognition models on a recently introduced dataset consisting of minimal pairs of aligned sentences, one serious, the other humorous. We find that, although our aligned dataset is much harder than previous datasets, transformer-based models recognize the humorous sentence in an aligned pair with high accuracy (78%). In a careful error analysis, we characterize easy vs hard instances. Finally, by analyzing attention weights, we obtain important insights into the mechanisms by which transformers recognize humor. Most remarkably, we find clear evidence that one single attention head learns to recognize the words that make a test sentence humorous, even without access to this information at training time.