An Empirical Study on the Generalization Power of Neural Representations Learned via Visual Guessing Games
Not on the map. Why?
Abstract (arXiv)
Guessing games are a prototypical instance of the "learning by interacting" paradigm. This work investigates how well an artificial agent can benefit from playing guessing games when later asked to perform on novel NLP downstream tasks such as Visual Question Answering (VQA). We propose two ways to exploit playing guessing games: 1) a supervised learning scenario in which the agent learns to mimic successful guessing games and 2) a novel way for an agent to play by itself, called Self-play via Iterated Experience Learning (SPIEL). We evaluate the ability of both procedures to generalize: an in-domain evaluation shows an increased accuracy (+7.79) compared with competitors on the evaluation suite CompGuessWhat?!; a transfer evaluation shows improved performance for VQA on the TDIUC dataset in terms of harmonic average accuracy (+5.31) thanks to more fine-grained object representations learned via SPIEL.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 41 references that are in the map. First posted on arXiv 31 Jan 2021.
Builds on
30 of 41 references.
References · 30
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2015VQA: Visual Question Answeringcited by 2,489
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
- 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
- 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,291
- 2015Prioritized Experience Replaycited by 1,070
- 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 668
- 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
- 2016Visual Dialogcited by 377
18 more
- 2015Neural Module Networkscited by 375
- 201912-in-1: Multi-Task Vision and Language Representation Learningcited by 278
- 2016Multimodal Compact Bilinear Pooling for Visual Question Answering and Visual Groundingcited by 243
- 2020Experience Grounds Languagecited by 241
- 2019UNITER: UNiversal Image-TExt Representation Learningcited by 225
- 2019TransferTransfo: A Transfer Learning Approach for Neural Network Based Conversational Agentscited by 188
- 2016GuessWhat?! Visual object discovery through multi-modal dialoguecited by 112
- 2017An Analysis of Visual Question Answering Algorithmscited by 90
- 2019MUREL: Multimodal Relational Reasoning for Visual Question Answeringcited by 68
- 2017Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queriescited by 56
- 2019Countering Language Drift via Visual Groundingcited by 55
- 2018A Hierarchical Multi-task Approach for Learning Embeddings from Semantic Taskscited by 54
- 2017End-to-end optimization of goal-driven and visually grounded dialogue systemscited by 17
- 2018Beyond task success: A closer look at jointly learning to see, ask, and GuessWhatcited by 15
- 2016Training Recurrent Answering Units with Joint Loss Minimization for VQAcited by 9
- 2020CompGuessWhat?!: A Multi-task Evaluation Framework for Grounded Language Learningcited by 8
- 2020Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Datacited by 7
- 2018Ask No More: Deciding when to guess in referential visual dialoguecited by 3
Built on by
Citing papers per year: 2023: 1.
1 paper.