# 3M: Multi-style image caption generation using Multi-modality features under Multi-UPDOWN model

arXiv: https://arxiv.org/abs/2103.11186 · HTML: https://mlsanity.com/paper/2103.11186/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 20 references that are in the map.
- First posted on arXiv 20 Mar 2021.

## Builds on (most cited first, 12 of 12)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [CIDEr: Consensus-based Image Description Evaluation](https://mlsanity.com/paper/1411.5726/) (2014)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [SPICE: Semantic Propositional Image Caption Evaluation](https://mlsanity.com/paper/1607.08822/) (2016)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [YFCC100M: The New Data in Multimedia Research](https://mlsanity.com/paper/1503.01817/) (2015)
- [DenseCap: Fully Convolutional Localization Networks for Dense Captioning](https://mlsanity.com/paper/1511.07571/) (2015)
- [Engaging Image Captioning Via Personality](https://mlsanity.com/paper/1810.10665/) (2018)
- [Regularizing Neural Machine Translation by Target-bidirectional Agreement](https://mlsanity.com/paper/1808.04064/) (2018)
- ["Factual" or "Emotional": Stylized Image Captioning with Adaptive Learning and Attention](https://mlsanity.com/paper/1807.03871/) (2018)

## Built on by (most cited first, 2 of 2)

- [Style-Aware Contrastive Learning for Multi-Style Image Captioning](https://mlsanity.com/paper/2301.11367/) (2023)
- [StyleM: Stylized Metrics for Image Captioning Built with Contrastive N-grams](https://mlsanity.com/paper/2201.00975/) (2022)

## Abstract (arXiv)

In this paper, we build a multi-style generative model for stylish image captioning which uses multi-modality image features, ResNeXt features and text features generated by DenseCap. We propose the 3M model, a Multi-UPDOWN caption model that encodes multi-modality features and decode them to captions. We demonstrate the effectiveness of our model on generating human-like captions by examining its performance on two datasets, the PERSONALITY-CAPTIONS dataset and the FlickrStyle10K dataset. We compare against a variety of state-of-the-art baselines on various automatic NLP metrics such as BLEU, ROUGE-L, CIDEr, SPICE, etc. A qualitative study has also been done to verify our 3M model can be used for generating different stylized captions.