Interactive Machine Learning for Image Captioning
Not on the map. Why?
Abstract (arXiv)
We propose an approach for interactive learning for an image captioning model. As human feedback is expensive and modern neural network based approaches often require large amounts of supervised data to be trained, we envision a system that exploits human feedback as good as possible by multiplying the feedback using data augmentation methods, and integrating the resulting training examples into the model in a smart way. This approach has three key components, for which we need to find suitable practical implementations: feedback collection, data augmentation, and model update. We outline our idea and review different possibilities to address these tasks.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 28 Feb 2022.
Builds on
27 of 32 references.
References · 27
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
- 2015Character-level Convolutional Networks for Text Classificationcited by 2,293
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
- 2019EDA: Easy Data Augmentation Techniques for Boosting Performance on Text Classification Taskscited by 800
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
- 2013An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networkscited by 711
- 2016Self-critical Sequence Training for Image Captioningcited by 544
- 2021A Survey of Data Augmentation Approaches for NLPcited by 408
- 2018Neural Baby Talkcited by 107
15 more
- 2016Guided Open Vocabulary Image Captioning with Constrained Beam Searchcited by 105
- 2018Deep Bayesian Active Learning for Natural Language Processing: Results of a Large-Scale Empirical Studycited by 90
- 2018Data Augmentation using Random Image Cropping and Patching for Deep CNNscited by 87
- 2020Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphscited by 87
- 2018Practical Obstacles to Deploying Active Learningcited by 68
- 2018Show, Control and Tell: A Framework for Generating Controllable and Grounded Captionscited by 65
- 2021Human-like Controllable Image Captioning with Verb-specific Semantic Rolescited by 41
- 2015Deep Compositional Captioning: Describing Novel Object Categories without Paired Training Datacited by 36
- 2020RATT: Recurrent Attention to Transient Tasks for Continual Image Captioningcited by 32
- 2018Intention Oriented Image Captions with Guiding Objectscited by 20
- 2019Atalaya at TASS 2019: Data Augmentation and Robust Embeddings for Sentiment Analysiscited by 12
- 2018Learning to Caption Images through a Lifetime by Asking Questionscited by 11
- 2017Teaching Machines to Describe Images via Natural Language Feedbackcited by 8
- 2021Similar Scenes arouse Similar Emotions: Parallel Data Augmentation for Stylized Image Captioningcited by 7
- 2021Image Captioning using Deep Stacked LSTMs, Contextual Word Embeddings and Data Augmentationcited by 5
Built on by
Citing papers per year: 2023: 3.
3 papers.