# Small But Funny: A Feedback-Driven Approach to Humor Distillation

arXiv: https://arxiv.org/abs/2402.18113 · HTML: https://mlsanity.com/paper/2402.18113/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 26 references that are in the map.
- First posted on arXiv 28 Feb 2024.

## Builds on (most cited first, 10 of 10)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Towards a Human-like Open-Domain Chatbot](https://mlsanity.com/paper/2001.09977/) (2020)
- [A Long Way to Go: Investigating Length Correlations in RLHF](https://mlsanity.com/paper/2310.03716/) (2023)
- [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023)
- [Leveraging Large Language Models for Multiple Choice Question Answering](https://mlsanity.com/paper/2210.12353/) (2022)
- [Branch-Solve-Merge Improves Large Language Model Evaluation and Generation](https://mlsanity.com/paper/2310.15123/) (2023)
- [Loose lips sink ships: Mitigating Length Bias in Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.05199/) (2023)

## Built on by (most cited first, 12 of 12)

- [What is the Role of Small Models in the LLM Era: A Survey](https://mlsanity.com/paper/2409.06857/) (2024)
- [SynthesizeMe! Inducing Persona-Guided Prompts for Personalized Reward Models in LLMs](https://mlsanity.com/paper/2506.05598/) (2025)
- ["A good pun is its own reword": Can Large Language Models Understand Puns?](https://mlsanity.com/paper/2404.13599/) (2024)
- [Small Language Models can Outperform Humans in Short Creative Writing: A Study Comparing SLMs with Humans and LLMs](https://mlsanity.com/paper/2409.11547/) (2024)
- [Creativity in AI: Progresses and Challenges](https://mlsanity.com/paper/2410.17218/) (2024)
- [Not All Jokes Land: Evaluating Large Language Models Understanding of Workplace Humor](https://mlsanity.com/paper/2506.01819/) (2025)
- [daDPO: Distribution-Aware DPO for Distilling Conversational Abilities](https://mlsanity.com/paper/2506.15717/) (2025)
- [CoDi: Conversational Distillation for Grounded Question Answering](https://mlsanity.com/paper/2408.11219/) (2024)
- [A Benchmark for Cross-Domain Argumentative Stance Classification on Social Media](https://mlsanity.com/paper/2410.08900/) (2024)
- [Multi-Agent Comedy Club: Investigating Community Discussion Effects on LLM Humor Generation](https://mlsanity.com/paper/2602.14770/) (2026)
- [HumorRank: A Tournament-Based Leaderboard for Evaluating Humor Generation in Large Language Models](https://mlsanity.com/paper/2604.19786/) (2026)
- [Towards High-Level Semantic Intelligence](https://mlsanity.com/paper/2607.24082/) (2026)

## Abstract (arXiv)

The emergence of Large Language Models (LLMs) has brought to light promising language generation capabilities, particularly in performing tasks like complex reasoning and creative writing. Consequently, distillation through imitation of teacher responses has emerged as a popular technique to transfer knowledge from LLMs to more accessible, Small Language Models (SLMs). While this works well for simpler tasks, there is a substantial performance gap on tasks requiring intricate language comprehension and creativity, such as humor generation. We hypothesize that this gap may stem from the fact that creative tasks might be hard to learn by imitation alone and explore whether an approach, involving supplementary guidance from the teacher, could yield higher performance. To address this, we study the effect of assigning a dual role to the LLM - as a "teacher" generating data, as well as a "critic" evaluating the student's performance. Our experiments on humor generation reveal that the incorporation of feedback significantly narrows the performance gap between SLMs and their larger counterparts compared to merely relying on imitation. As a result, our research highlights the potential of using feedback as an additional dimension to data when transferring complex language abilities via distillation.