Neural Variational Learning for Grounded Language Acquisition
Not on the map. Why?
Abstract (arXiv)
We propose a learning system in which language is grounded in visual percepts without specific pre-defined categories of terms. We present a unified generative method to acquire a shared semantic/visual embedding that enables the learning of language about a wide range of real-world objects. We evaluate the efficacy of this learning by predicting the semantics of objects and comparing the performance with neural and non-neural inputs. We show that this generative approach exhibits promising results in language grounding without pre-specifying visual categories under low resource settings. Our experiments demonstrate that this approach is generalizable to multilingual, highly varied datasets.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 63 references that are in the map. First posted on arXiv 20 Jul 2021.
Builds on
30 of 63 references.
References · 30
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
- 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
- 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
- 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
- 2015VQA: Visual Question Answeringcited by 2,489
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
- 2017Learning Transferable Architectures for Scalable Image Recognitioncited by 1,097
- 2014Semi-Supervised Learning with Deep Generative Modelscited by 494
- 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
- 2020Experience Grounds Languagecited by 241
18 more
- 2020SEED: Semantics Enhanced Encoder-Decoder Framework for Scene Text Recognitioncited by 119
- 2015Grounding of Textual Phrases in Images by Reconstructioncited by 112
- 2017Text2Action: Generative Adversarial Synthesis from Language to Actioncited by 106
- 2020MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioningcited by 79
- 2017Variational Autoencoders for Deforming 3D Mesh Modelscited by 69
- 2017Learning a bidirectional mapping between human whole-body motion and natural language using deep recurrent neural networkscited by 61
- 2017Interactively Picking Real-World Objects with Unconstrained Spoken Language Instructionscited by 61
- 2020Reusing Discriminators for Encoding: Towards Unsupervised Image-to-Image Translationcited by 60
- 2019Just Ask:An Interactive Learning Framework for Vision and Language Navigationcited by 53
- 2018Object Referring in Videos with Language and Human Gazecited by 40
- 2020Learning the Redundancy-free Features for Generalized Zero-Shot Object Recognitioncited by 34
- 2020Words aren't enough, their order matters: On the Robustness of Grounding Visual Referring Expressionscited by 33
- 2015Symbol Emergence in Robotics: A Surveycited by 31
- 2020Robot Object Retrieval with Contextual Natural Language Queriescited by 26
- 2021Task Aligned Generative Meta-learning for Zero-shot Learningcited by 19
- 2017Crossing Nets: Combining GANs and VAEs with a Shared Latent Space for Hand Pose Estimationcited by 13
- 2019Hierarchical Photo-Scene Encoder for Album Storytellingcited by 5
- 2018Mix and match networks: encoder-decoder alignment for zero-pair image translationcited by 3
Built on by
Citing papers per year: 2023: 1.
1 paper.