Jointly Learning Truth-Conditional Denotations and Groundings using Parallel Attention
Not on the map. Why?
Abstract (arXiv)
We present a model that jointly learns the denotations of words together with their groundings using a truth-conditional semantics. Our model builds on the neurosymbolic approach of Mao et al. (2019), learning to ground objects in the CLEVR dataset (Johnson et al., 2017) using a novel parallel attention mechanism. The model achieves state of the art performance on visual question answering, learning to detect and ground objects with question performance as the only training signal. We also show that the model is able to learn flexible non-canonical groundings just by adjusting answers to questions in the training set.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 21 of its 37 references that are in the map. First posted on arXiv 14 Apr 2021.
Builds on
21 of 37 references.
References · 21
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2015VQA: Visual Question Answeringcited by 2,506
- 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
- 2016CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoningcited by 1,457
- 2020Object-Centric Learning with Slot Attentioncited by 876
- 2019MONet: Unsupervised Scene Decomposition and Representationcited by 392
- 2015Neural Module Networkscited by 376
- 2019The Neuro-Symbolic Concept Learner: Interpreting Scenes, Words, and Sentences From Natural Supervisioncited by 372
- 2019Multi-Object Representation Learning with Iterative Variational Inferencecited by 360
- 2015MovieQA: Understanding Stories in Movies through Question-Answeringcited by 304
- 2020AdaBelief Optimizer: Adapting Stepsizes by the Belief in Observed Gradientscited by 283
- 2019GENESIS: Generative Scene Inference and Sampling with Object-Centric Latent Representationscited by 227
9 more
- 2020SPACE: Unsupervised Object-Oriented Scene Representation via Spatial Attention and Decompositioncited by 190
- 2018Compositional Attention Networks for Machine Reasoningcited by 183
- 2017Inferring and Executing Programs for Visual Reasoningcited by 130
- 2019SCALOR: Generative World Models with Scalable Object Representationscited by 108
- 2017Grounded Language Learning in a Simulated 3D Worldcited by 100
- 2016Tagger: Deep Unsupervised Perceptual Groupingcited by 69
- 2019CLOSURE: Assessing Systematic Generalization of CLEVR Modelscited by 63
- 2018Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoningcited by 51
- 2017Generalized Grounding Graphs: A Probabilistic Framework for Understanding Grounded Commandscited by 4
Built on by
Citing papers per year: 2022: 1.
1 paper.