LAVA: Label-efficient Visual Learning and Adaptation
Not on the map. Why?
Abstract (arXiv)
We present LAVA, a simple yet effective method for multi-domain visual transfer learning with limited data. LAVA builds on a few recent innovations to enable adapting to partially labelled datasets with class and domain shifts. First, LAVA learns self-supervised visual representations on the source dataset and ground them using class label semantics to overcome transfer collapse problems associated with supervised pretraining. Secondly, LAVA maximises the gains from unlabelled target data via a novel method which uses multi-crop augmentations to obtain highly robust pseudo-labels. By combining these ingredients, LAVA achieves a new state-of-the-art on ImageNet semi-supervised protocol, as well as on 7 out of 10 datasets in multi-domain few-shot learning on the Meta-dataset. Code and models are made available.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 50 of its 59 references that are in the map. First posted on arXiv 19 Oct 2022.
Builds on
50 of 59 references.
References · 50
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
- 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,346
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
38 more
- 2020Unsupervised Learning of Visual Features by Contrasting Cluster Assignmentscited by 2,845
- 2020Supervised Contrastive Learningcited by 2,803
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2020FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidencecited by 2,080
- 2019MixMatch: A Holistic Approach to Semi-Supervised Learningcited by 1,390
- 2020Big Self-Supervised Models are Strong Semi-Supervised Learnerscited by 1,375
- 2018Moment Matching for Multi-Source Domain Adaptationcited by 1,327
- 2019Self-training with Noisy Student improves ImageNet classificationcited by 1,304
- 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
- 2016ConceptNet 5.5: An Open Multilingual Graph of General Knowledgecited by 1,065
- 2019Unsupervised Data Augmentation for Consistency Trainingcited by 1,057
- 2020MPNet: Masked and Permuted Pre-training for Language Understandingcited by 973
- 2017Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learningcited by 918
- 2019Learning Representations by Maximizing Mutual Information Across Viewscited by 733
- 2018Meta-Learning with Latent Embedding Optimizationcited by 476
- 2017Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning resultscited by 470
- 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
- 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
- 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
- 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
- 2018Meta-learning with differentiable closed-form solverscited by 395
- 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
- 2020Meta Pseudo Labelscited by 372
- 2019ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoringcited by 349
- 2019S4L: Self-Supervised Semi-Supervised Learningcited by 335
- 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
- 2019Interpolation Consistency Training for Semi-Supervised Learningcited by 248
- 2020CrossTransformers: spatially-aware few-shot transfercited by 174
- 2018Task-Agnostic Meta-Learning for Few-shot Learningcited by 151
- 2021Efficient Self-supervised Vision Transformers for Representation Learningcited by 149
- 2021Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samplescited by 114
- 2020Meta-Learning with Adaptive Hyperparameterscited by 79
- 2020A Universal Representation Transformer Layer for Few-Shot Image Classificationcited by 65
- 2019Few-Shot Learning with Embedded Class Models and Shot-Free Meta Trainingcited by 57
- 2021All Labels Are Not Created Equal: Enhancing Semi-supervision via Label Grouping and Co-trainingcited by 32
- 2020Optimized Generic Feature Learning for Few-shot Classification across Domainscited by 23
- 2020Can Semantic Labels Assist Self-Supervised Visual Representation Learning?cited by 16
- 2021Rich Semantics Improve Few-shot Learningcited by 15
Built on by
Citing papers per year: 2023: 1.
1 paper.