# We Are Humor Beings: Understanding and Predicting Visual Humor

arXiv: https://arxiv.org/abs/1512.04407 · HTML: https://mlsanity.com/paper/1512.04407/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2015.

## Built on by (most cited first, 15 of 15)

- [Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest](https://mlsanity.com/paper/2209.06293/) (2022)
- [Multimodal Learning using Optimal Transport for Sarcasm and Humor Detection](https://mlsanity.com/paper/2110.10949/) (2021)
- [Can Language Models Laugh at YouTube Short-form Videos?](https://mlsanity.com/paper/2310.14159/) (2023)
- [Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions](https://mlsanity.com/paper/2405.19088/) (2024)
- [Systematic Literature Review: Computational Approaches for Humour Style Classification](https://mlsanity.com/paper/2402.01759/) (2024)
- [HumorDB: Can AI understand graphical humor?](https://mlsanity.com/paper/2406.13564/) (2024)
- [BottleHumor: Self-Informed Humor Explanation using the Information Bottleneck Principle](https://mlsanity.com/paper/2502.18331/) (2025)
- [Humor in Pixels: Benchmarking Large Multimodal Models Understanding of Online Comics](https://mlsanity.com/paper/2509.12248/) (2025)
- [Seeing the Intangible: Survey of Image Classification into High-Level and Abstract Categories](https://mlsanity.com/paper/2308.10562/) (2023)
- [Looking Beyond the Obvious: A Survey on Abstract Concept Recognition for Video Understanding](https://mlsanity.com/paper/2508.20765/) (2025)
- [Predicting Winning Captions for Weekly New Yorker Comics](https://mlsanity.com/paper/2407.18949/) (2024)
- [v-HUB: A Benchmark for Video Humor Understanding from Vision and Sound](https://mlsanity.com/paper/2509.25773/) (2025)
- [Robusto-1 Dataset: Comparing Humans and VLMs on real out-of-distribution Autonomous Driving VQA from Peru](https://mlsanity.com/paper/2503.07587/) (2025)
- [Grounded Satirical Generation with RAG](https://mlsanity.com/paper/2605.10853/) (2026)
- [Computational Humor with Multimodal LLMs: Methods, Datasets, Evaluation, and Challenges](https://mlsanity.com/paper/2607.19011/) (2026)

## Abstract (arXiv)

Humor is an integral part of human lives. Despite being tremendously impactful, it is perhaps surprising that we do not have a detailed understanding of humor yet. As interactions between humans and AI systems increase, it is imperative that these systems are taught to understand subtleties of human expressions such as humor. In this work, we are interested in the question - what content in a scene causes it to be funny? As a first step towards understanding visual humor, we analyze the humor manifested in abstract scenes and design computational models for them. We collect two datasets of abstract scenes that facilitate the study of humor at both the scene-level and the object-level. We analyze the funny scenes and explore the different types of humor depicted in them via human studies. We model two tasks that we believe demonstrate an understanding of some aspects of visual humor. The tasks involve predicting the funniness of a scene and altering the funniness of a scene. We show that our models perform well quantitatively, and qualitatively through human studies. Our datasets are publicly available.