# A Robot Walks into a Bar: Can Language Models Serve as Creativity Support Tools for Comedy? An Evaluation of LLMs' Humour Alignment with Comedians

arXiv: https://arxiv.org/abs/2405.20956 · HTML: https://mlsanity.com/paper/2405.20956/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 113 references that are in the map.
- First posted on arXiv 31 May 2024.

## Builds on (most cited first, 30 of 44)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [PaLM: Scaling Language Modeling with Pathways](https://mlsanity.com/paper/2204.02311/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Fine-Tuning Language Models from Human Preferences](https://mlsanity.com/paper/1909.08593/) (2019)
- [Model Cards for Model Reporting](https://mlsanity.com/paper/1810.03993/) (2018)
- [A General Language Assistant as a Laboratory for Alignment](https://mlsanity.com/paper/2112.00861/) (2021)
- [Ethical and social risks of harm from Language Models](https://mlsanity.com/paper/2112.04359/) (2021)
- [Whose Opinions Do Language Models Reflect?](https://mlsanity.com/paper/2303.17548/) (2023)
- [Artificial Intelligence, Values and Alignment](https://mlsanity.com/paper/2001.09768/) (2020)
- [Persistent Anti-Muslim Bias in Large Language Models](https://mlsanity.com/paper/2101.05783/) (2021)
- [Societal Biases in Language Generation: Progress and Challenges](https://mlsanity.com/paper/2105.04054/) (2021)
- [Does Writing with Language Models Reduce Content Diversity?](https://mlsanity.com/paper/2309.05196/) (2023)
- [Re3: Generating Longer Stories With Recursive Reprompting and Revision](https://mlsanity.com/paper/2210.06774/) (2022)
- [Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals](https://mlsanity.com/paper/2209.14958/) (2022)
- [Fine-tuning language models to find agreement among humans with diverse preferences](https://mlsanity.com/paper/2211.15006/) (2022)
- [Art or Artifice? Large Language Models and the False Promise of Creativity](https://mlsanity.com/paper/2309.14556/) (2023)
- [Probing Pre-Trained Language Models for Cross-Cultural Differences in Values](https://mlsanity.com/paper/2203.13722/) (2022)
- [Cultural Alignment in Large Language Models: An Explanatory Analysis Based on Hofstede's Cultural Dimensions](https://mlsanity.com/paper/2309.12342/) (2023)
- [Generative Language Models Exhibit Social Identity Biases](https://mlsanity.com/paper/2310.15819/) (2023)
- [Detoxifying Language Models Risks Marginalizing Minority Voices](https://mlsanity.com/paper/2104.06390/) (2021)
- [Decolonial AI: Decolonial Theory as Sociotechnical Foresight in Artificial Intelligence](https://mlsanity.com/paper/2007.04068/) (2020)
- [On Measures of Biases and Harms in NLP](https://mlsanity.com/paper/2108.03362/) (2021)
- [Sparks: Inspiration for Science Writing using Language Models](https://mlsanity.com/paper/2110.07640/) (2021)
- [Help me write a poem: Instruction Tuning as a Vehicle for Collaborative Poetry Writing](https://mlsanity.com/paper/2210.13669/) (2022)
- [Creative Writing with an AI-Powered Writing Assistant: Perspectives from Professional Writers](https://mlsanity.com/paper/2211.05030/) (2022)
- [In conversation with Artificial Intelligence: aligning language models with human values](https://mlsanity.com/paper/2209.00731/) (2022)
- [AI's Regimes of Representation: A Community-centered Study of Text-to-Image Models in South Asia](https://mlsanity.com/paper/2305.11844/) (2023)
- [The Past, Present and Better Future of Feedback Learning in Large Language Models for Subjective Human Preferences and Values](https://mlsanity.com/paper/2310.07629/) (2023)

## Built on by (most cited first, 29 of 29)

- [Can AI writing be salvaged? Mitigating Idiosyncrasies and Improving Human-AI Alignment in the Writing Process through Edits](https://mlsanity.com/paper/2409.14509/) (2024)
- [Roll the dice & look before you leap: Going beyond the creative limits of next-token prediction](https://mlsanity.com/paper/2504.15266/) (2025)
- [AI-Slop to AI-Polish? Aligning Language Models through Edit-Based Writing Rewards and Test-time Computation](https://mlsanity.com/paper/2504.07532/) (2025)
- [CS4: Measuring the Creativity of Large Language Models Automatically by Controlling the Number of Story-Writing Constraints](https://mlsanity.com/paper/2410.04197/) (2024)
- [Which Contributions Deserve Credit? Perceptions of Attribution in Human-AI Co-Creation](https://mlsanity.com/paper/2502.18357/) (2025)
- [Guardians and Offenders: A Survey on Harmful Content Generation and Safety Mitigation of LLM](https://mlsanity.com/paper/2508.05775/) (2025)
- [Large Language Models for Subjective Language Understanding: A Survey](https://mlsanity.com/paper/2508.07959/) (2025)
- [Can Good Writing Be Generative? Expert-Level AI Writing Emerges through Fine-Tuning on High-Quality Books](https://mlsanity.com/paper/2601.18353/) (2026)
- [Un-Straightening Generative AI: How Queer Artists Surface and Challenge the Normativity of Generative AI Models](https://mlsanity.com/paper/2503.09805/) (2025)
- [CleanComedy: Creating Friendly Humor through Generative Techniques](https://mlsanity.com/paper/2412.09203/) (2024)
- [Humor in Pixels: Benchmarking Large Multimodal Models Understanding of Online Comics](https://mlsanity.com/paper/2509.12248/) (2025)
- [The Theater Stage as Laboratory: Review of Real-Time Comedy LLM Systems for Live Performance](https://mlsanity.com/paper/2501.08474/) (2025)
- [Expertise elevates AI usage: experimental evidence comparing laypeople and professional artists](https://mlsanity.com/paper/2501.12374/) (2025)
- [Constella: Supporting Storywriters' Interconnected Character Creation through LLM-based Multi-Agents](https://mlsanity.com/paper/2507.05820/) (2025)
- [Evaluating the Creativity of LLMs in Persian Literary Text Generation](https://mlsanity.com/paper/2509.18401/) (2025)
- [Pun Unintended: LLMs and the Illusion of Humor Understanding](https://mlsanity.com/paper/2509.12158/) (2025)
- [Addressing LLM Diversity by Infusing Random Concepts](https://mlsanity.com/paper/2601.18053/) (2026)
- [On the Wings of Imagination: Conflicting Script-based Multi-role Framework for Humor Caption Generation](https://mlsanity.com/paper/2602.06423/) (2026)
- [ACAI for SBOs: AI Co-creation for Advertising and Inspiration for Small Business Owners](https://mlsanity.com/paper/2503.06729/) (2025)
- [Do Chinese models speak Chinese languages?](https://mlsanity.com/paper/2504.00289/) (2025)
- [Governing AI Beyond the Pretraining Frontier](https://mlsanity.com/paper/2502.15719/) (2025)
- [Bypassing Safety Guardrails in LLMs Using Humor](https://mlsanity.com/paper/2504.06577/) (2025)
- [Accepted with Minor Revisions: Value of AI-Assisted Scientific Writing](https://mlsanity.com/paper/2511.12529/) (2025)
- [Timing In stand-up Comedy: Text, Audio, Laughter, Kinesics (TIC-TALK): Pipeline and Database for the Multimodal Study of Comedic Timing](https://mlsanity.com/paper/2603.21803/) (2026)
- [Dialogue with the Machine and Dialogue with the Art World: Evaluating Generative AI for Culturally-Situated Creativity](https://mlsanity.com/paper/2412.14077/) (2024)
- [Decoders Laugh as Loud as Encoders](https://mlsanity.com/paper/2509.04779/) (2025)
- ["Not Human, Funnier": How Machine Identity Shapes Humor Perception in Online AI Stand-up Comedy](https://mlsanity.com/paper/2602.12763/) (2026)
- [Investigating Counterfactual Unfairness in LLMs towards Identities through Humor](https://mlsanity.com/paper/2604.18729/) (2026)
- [CASPER in the Machine: Insights into Character Variety in LLM-Generated Stories](https://mlsanity.com/paper/2606.22454/) (2026)

## Abstract (arXiv)

We interviewed twenty professional comedians who perform live shows in front of audiences and who use artificial intelligence in their artistic process as part of 3-hour workshops on ``AI x Comedy'' conducted at the Edinburgh Festival Fringe in August 2023 and online. The workshop consisted of a comedy writing session with large language models (LLMs), a human-computer interaction questionnaire to assess the Creativity Support Index of AI as a writing tool, and a focus group interrogating the comedians' motivations for and processes of using AI, as well as their ethical concerns about bias, censorship and copyright. Participants noted that existing moderation strategies used in safety filtering and instruction-tuned LLMs reinforced hegemonic viewpoints by erasing minority groups and their perspectives, and qualified this as a form of censorship. At the same time, most participants felt the LLMs did not succeed as a creativity support tool, by producing bland and biased comedy tropes, akin to ``cruise ship comedy material from the 1950s, but a bit less racist''. Our work extends scholarship about the subtle difference between, one the one hand, harmful speech, and on the other hand, ``offensive'' language as a practice of resistance, satire and ``punching up''. We also interrogate the global value alignment behind such language models, and discuss the importance of community-based value alignment and data ownership to build AI tools that better suit artists' needs.