# Predicting Winning Captions for Weekly New Yorker Comics

arXiv: https://arxiv.org/abs/2407.18949 · HTML: https://mlsanity.com/paper/2407.18949/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 26 references that are in the map.
- First posted on arXiv 12 Jul 2024.

## Builds on (most cited first, 15 of 15)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [LoftQ: LoRA-Fine-Tuning-Aware Quantization for Large Language Models](https://mlsanity.com/paper/2310.08659/) (2023)
- [Do Androids Laugh at Electric Sheep? Humor "Understanding" Benchmarks from The New Yorker Caption Contest](https://mlsanity.com/paper/2209.06293/) (2022)
- [Can Vision Transformers Learn without Natural Images?](https://mlsanity.com/paper/2103.13023/) (2021)
- [Humor in Collective Discourse: Unsupervised Funniness Detection in the New Yorker Cartoon Caption Contest](https://mlsanity.com/paper/1506.08126/) (2015)
- [We Are Humor Beings: Understanding and Predicting Visual Humor](https://mlsanity.com/paper/1512.04407/) (2015)
- [What a Whole Slide Image Can Tell? Subtype-guided Masked Transformer for Pathological Image Captioning](https://mlsanity.com/paper/2310.20607/) (2023)
- [Enhancing image captioning with depth information using a Transformer-based framework](https://mlsanity.com/paper/2308.03767/) (2023)

## Built on by (most cited first, 1 of 1)

- [Have Large Vision-Language Models Mastered Art History?](https://mlsanity.com/paper/2409.03521/) (2024)

## Abstract (arXiv)

Image captioning using Vision Transformers (ViTs) represents a pivotal convergence of computer vision and natural language processing, offering the potential to enhance user experiences, improve accessibility, and provide textual representations of visual data. This paper explores the application of image captioning techniques to New Yorker cartoons, aiming to generate captions that emulate the wit and humor of winning entries in the New Yorker Cartoon Caption Contest. This task necessitates sophisticated visual and linguistic processing, along with an understanding of cultural nuances and humor. We propose several new baselines for using vision transformer encoder-decoder models to generate captions for the New Yorker cartoon caption contest.