MetaReVision: Meta-Learning with Retrieval for Visually Grounded Compositional Concept Acquisition
Not on the map. Why?
Abstract (arXiv)
Humans have the ability to learn novel compositional concepts by recalling and generalizing primitive concepts acquired from past experiences. Inspired by this observation, in this paper, we propose MetaReVision, a retrieval-enhanced meta-learning model to address the visually grounded compositional concept learning problem. The proposed MetaReVision consists of a retrieval module and a meta-learning module which are designed to incorporate retrieved primitive concepts as a supporting set to meta-train vision-anguage models for grounded compositional concept recognition. Through meta-learning from episodes constructed by the retriever, MetaReVision learns a generic compositional representation that can be fast updated to recognize novel compositional concepts. We create CompCOCO and CompFlickr to benchmark the grounded compositional concept learning. Our experimental results show that MetaReVision outperforms other competitive baselines and the retrieval module plays an important role in this compositional learning process.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 33 of its 46 references that are in the map. First posted on arXiv 2 Nov 2023.
Builds on
33 of 46 references.
References · 33
- 2014Adam: A Method for Stochastic Optimizationcited by 37,240
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
- 2017Prototypical Networks for Few-shot Learningcited by 3,015
- 2017Billion-scale similarity search with GPUscited by 2,497
- 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
- 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
- 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
- 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
- 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,143
- 2018On First-Order Meta-Learning Algorithmscited by 1,018
- 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
21 more
- 2019Generalization through Memorization: Nearest Neighbor Language Modelscited by 800
- 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 723
- 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
- 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
- 2015Neural Module Networkscited by 376
- 2019Compositionality decomposed: how do neural networks generalise?cited by 265
- 2017Feature Generating Networks for Zero-Shot Learningcited by 257
- 2020Towards Learning a Generic Agent for Vision-and-Language Navigation via Pre-trainingcited by 231
- 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 181
- 2022CLIP Models are Few-shot Learners: Empirical Studies on VQA and Visual Entailmentcited by 113
- 2020A Benchmark for Systematic Generalization in Grounded Language Understandingcited by 108
- 2018Neural Baby Talkcited by 107
- 2022Retrieval Augmented Classification for Long-Tail Visual Recognitioncited by 106
- 2019Meta-Learning without Memorizationcited by 92
- 2021Improving Named Entity Recognition by External Context Retrieving and Cooperative Learningcited by 76
- 2021Making Transformers Solve Compositional Taskscited by 72
- 2021Meta-Learning to Compositionally Generalizecited by 62
- 2022Retrieval-Augmented Reinforcement Learningcited by 53
- 2021Systematic Generalization with Edge Transformerscited by 38
- 2019Compositional Generalization in Image Captioningcited by 23
- 2022WinoDict: Probing language models for in-context word acquisitioncited by 15
Built on by
Citing papers per year: 2024: 2.
2 papers.