# ChatGPT is fun, but it is not funny! Humor is still challenging Large Language Models

arXiv: https://arxiv.org/abs/2306.04563 · HTML: https://mlsanity.com/paper/2306.04563/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 59 references that are in the map.
- First posted on arXiv 7 Jun 2023.

## Builds on (most cited first, 12 of 12)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://mlsanity.com/paper/2204.06745/) (2022)
- [Large Pre-trained Language Models Contain Human-like Biases of What is Right and Wrong to Do](https://mlsanity.com/paper/2103.11790/) (2021)
- [Humor Detection: A Transformer Gets the Last Laugh](https://mlsanity.com/paper/1909.00252/) (2019)
- [Federated Learning for Personalized Humor Recognition](https://mlsanity.com/paper/2012.01675/) (2020)
- [DeHumor: Visual Analytics for Decomposing Humor](https://mlsanity.com/paper/2107.08356/) (2021)

## Built on by (most cited first, 24 of 24)

- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [SteerLM: Attribute Conditioned SFT as an (User-Steerable) Alternative to RLHF](https://mlsanity.com/paper/2310.05344/) (2023)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [A Confederacy of Models: a Comprehensive Evaluation of LLMs on Creative Writing](https://mlsanity.com/paper/2310.08433/) (2023)
- [GPTEval: A Survey on Assessments of ChatGPT and GPT-4](https://mlsanity.com/paper/2308.12488/) (2023)
- [Evaluating the Diversity and Quality of LLM Generated Content](https://mlsanity.com/paper/2504.12522/) (2025)
- [Probing the Creativity of Large Language Models: Can models produce divergent semantic association?](https://mlsanity.com/paper/2310.11158/) (2023)
- [Aligning Large Language Models with Human: A Survey](https://mlsanity.com/paper/2307.12966/) (2023)
- [Humor in AI: Massive Scale Crowd-Sourced Preferences and Benchmarks for Cartoon Captioning](https://mlsanity.com/paper/2406.10522/) (2024)
- [Do Large Language Models Understand Conversational Implicature -- A case study with a chinese sitcom](https://mlsanity.com/paper/2404.19509/) (2024)
- [Cracking the Code of Juxtaposition: Can AI Models Understand the Humorous Contradictions](https://mlsanity.com/paper/2405.19088/) (2024)
- [Prompting a Large Language Model to Generate Diverse Motivational Messages: A Comparison with Human-Written Messages](https://mlsanity.com/paper/2308.13479/) (2023)
- [Large Models of What? Mistaking Engineering Achievements for Human Linguistic Agency](https://mlsanity.com/paper/2407.08790/) (2024)
- [Diversity-Enhanced Reasoning for Subjective Questions](https://mlsanity.com/paper/2507.20187/) (2025)
- [When 'YES' Meets 'BUT': Can Large Models Comprehend Contradictory Humor Through Comparative Reasoning?](https://mlsanity.com/paper/2503.23137/) (2025)
- [On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation](https://mlsanity.com/paper/2602.06423/) (2026)
- [The Unlikely Duel: Evaluating Creative Writing in LLMs through a Unique Scenario](https://mlsanity.com/paper/2406.15891/) (2024)
- [Incorporating Different Verbal Cues to Improve Text-Based Computer-Delivered Health Messaging](https://mlsanity.com/paper/2404.13633/) (2024)
- [A Survey on the Real Power of ChatGPT](https://mlsanity.com/paper/2405.00704/) (2024)
- [JaccDiv: A Metric and Benchmark for Quantifying Diversity of Generated Marketing Text in the Music Industry](https://mlsanity.com/paper/2504.20849/) (2025)
- ["Not Human, Funnier": How Machine Identity Shapes Humor Perception in Online AI Stand-up Comedy](https://mlsanity.com/paper/2602.12763/) (2026)
- [Reasoning or Memorization: Can LLMs Understand and Generate Chinese Xiehouyu Riddles?](https://mlsanity.com/paper/2607.23440/) (2026)
- [CaRGo-T: Causal Reasoning Graph-of-Thought improves Multimodal Humor Comprehension](https://mlsanity.com/paper/2608.23172/) (2026)
- [Comedic Fool's Gold: Reward Exploits and Countermeasures in Conversational Humor](https://mlsanity.com/paper/2610.00197/) (2026)

## Abstract (arXiv)

Humor is a central aspect of human communication that has not been solved for artificial agents so far. Large language models (LLMs) are increasingly able to capture implicit and contextual information. Especially, OpenAI's ChatGPT recently gained immense public attention. The GPT3-based model almost seems to communicate on a human level and can even tell jokes. Humor is an essential component of human communication. But is ChatGPT really funny? We put ChatGPT's sense of humor to the test. In a series of exploratory experiments around jokes, i.e., generation, explanation, and detection, we seek to understand ChatGPT's capability to grasp and reproduce human humor. Since the model itself is not accessible, we applied prompt-based experiments. Our empirical evidence indicates that jokes are not hard-coded but mostly also not newly generated by the model. Over 90% of 1008 generated jokes were the same 25 Jokes. The system accurately explains valid jokes but also comes up with fictional explanations for invalid jokes. Joke-typical characteristics can mislead ChatGPT in the classification of jokes. ChatGPT has not solved computational humor yet but it can be a big leap toward "funny" machines.