# Captioning Images with Novel Objects via Online Vocabulary Expansion

arXiv: https://arxiv.org/abs/2003.03305 · HTML: https://mlsanity.com/paper/2003.03305/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Mar 2020.

## Built on by (most cited first, 2 of 2)

- [NOC-REK: Novel Object Captioning with Retrieved Vocabulary from External Knowledge](https://mlsanity.com/paper/2203.14499/) (2022)
- [RCA-NOC: Relative Contrastive Alignment for Novel Object Captioning](https://mlsanity.com/paper/2312.06299/) (2023)

## Abstract (arXiv)

In this study, we introduce a low cost method for generating descriptions from images containing novel objects. Generally, constructing a model, which can explain images with novel objects, is costly because of the following: (1) collecting a large amount of data for each category, and (2) retraining the entire system. If humans see a small number of novel objects, they are able to estimate their properties by associating their appearance with known objects. Accordingly, we propose a method that can explain images with novel objects without retraining using the word embeddings of the objects estimated from only a small number of image features of the objects. The method can be integrated with general image-captioning models. The experimental results show the effectiveness of our approach.