# Learning like a Child: Fast Novel Visual Concept Learning from Sentence Descriptions of Images

arXiv: https://arxiv.org/abs/1504.06692 · HTML: https://mlsanity.com/paper/1504.06692/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Apr 2015.

## Built on by (most cited first, 10 of 10)

- [Act Like a Radiologist: Towards Reliable Multi-view Correspondence Reasoning for Mammogram Mass Detection](https://mlsanity.com/paper/2105.10160/) (2021)
- [Sample-Efficient Learning of Novel Visual Concepts](https://mlsanity.com/paper/2306.09482/) (2023)
- [Unpaired Image Captioning by Image-level Weakly-Supervised Visual Concept Recognition](https://mlsanity.com/paper/2203.03195/) (2022)
- [Geometry Attention Transformer with Position-aware LSTMs for Image Captioning](https://mlsanity.com/paper/2110.00335/) (2021)
- [Benchmarking and Enhancing Disentanglement in Concept-Residual Models](https://mlsanity.com/paper/2312.00192/) (2023)
- [GC-TTS: Few-shot Speaker Adaptation with Geometric Constraints](https://mlsanity.com/paper/2108.06890/) (2021)
- [Goal-driven text descriptions for images](https://mlsanity.com/paper/2108.12575/) (2021)
- [A Deep Neural Framework for Image Caption Generation Using GRU-Based Attention Mechanism](https://mlsanity.com/paper/2203.01594/) (2022)
- [The Construction of Reality in an AI: A Review](https://mlsanity.com/paper/2302.05448/) (2023)
- [A Comparative Study of Pre-trained CNNs and GRU-Based Attention for Image Caption Generation](https://mlsanity.com/paper/2310.07252/) (2023)

## Abstract (arXiv)

In this paper, we address the task of learning novel visual concepts, and their interactions with other concepts, from a few images with sentence descriptions. Using linguistic context and visual features, our method is able to efficiently hypothesize the semantic meaning of new words and add them to its word dictionary so that they can be used to describe images which contain these novel concepts. Our method has an image captioning module based on m-RNN with several improvements. In particular, we propose a transposed weight sharing scheme, which not only improves performance on image captioning, but also makes the model more suitable for the novel concept learning task. We propose methods to prevent overfitting the new concepts. In addition, three novel concept datasets are constructed for this new task. In the experiments, we show that our method effectively learns novel visual concepts from a few examples without disturbing the previously learned concepts. The project page is http://www.stat.ucla.edu/~junhua.mao/projects/child_learning.html