Task Attribute Distance for Few-Shot Learning: Theoretical Analysis and Applications
Not on the map. Why?
Abstract (arXiv)
Few-shot learning (FSL) aims to learn novel tasks with very few labeled samples by leveraging experience from related training tasks. In this paper, we try to understand FSL by delving into two key questions: (1) How to quantify the relationship between training and novel tasks? (2) How does the relationship affect the adaptation difficulty on novel tasks for different models? To answer the two questions, we introduce Task Attribute Distance (TAD) built upon attributes as a metric to quantify the task relatedness. Unlike many existing metrics, TAD is model-agnostic, making it applicable to different FSL models. Then, we utilize TAD metric to establish a theoretical connection between task relatedness and task adaptation difficulty. By deriving the generalization error bound on a novel task, we discover how TAD measures the adaptation difficulty on novel tasks for FSL models. To validate our TAD metric and theoretical findings, we conduct experiments on three benchmarks. Our experimental results confirm that TAD metric effectively quantifies the task relatedness and reflects the adaptation difficulty on novel tasks for various FSL methods, even if some of them do not learn attributes explicitly or human-annotated attributes are not available. Finally, we present two applications of the proposed TAD metric: data augmentation and test-time intervention, which further verify its effectiveness and general applicability. The source code is available at https://github.com/hu-my/TaskAttributeDistance.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 68 of its 91 references that are in the map. First posted on arXiv 6 Mar 2024.
Builds on
68 of 91 references.
References · 68
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2017Prototypical Networks for Few-shot Learningcited by 3,001
- 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
- 2021Learning to Prompt for Vision-Language Modelscited by 2,269
- 2016Matching Networks for One Shot Learningcited by 2,246
- 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
- 2020Concept Bottleneck Modelscited by 1,022
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
38 more
- 2019A Closer Look at Few-shot Classificationcited by 679
- 2021Multimodal Few-Shot Learning with Frozen Language Modelscited by 657
- 2018Taskonomy: Disentangling Task Transfer Learningcited by 631
- 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
- 2017Meta-SGD: Learning to Learn Quickly for Few-Shot Learningcited by 377
- 2019Meta-Dataset: A Dataset of Datasets for Learning to Learn from Few Examplescited by 312
- 2018How to train your MAMLcited by 306
- 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
- 2017Data Augmentation Generative Adversarial Networkscited by 278
- 2019Cross Attention Network for Few-shot Classificationcited by 218
- 2020Few-Shot Learning via Learning the Representation, Provablycited by 213
- 2019Task2Vec: Task Embedding for Meta-Learningcited by 197
- 2020Cross-Domain Few-Shot Classification via Learned Feature-Wise Transformationcited by 183
- 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
- 2021Free Lunch for Few-shot Learning: Distribution Calibrationcited by 165
- 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 162
- 2020Provable Meta-Learning of Linear Representationscited by 161
- 2022Pushing the Limits of Simple Pipelines for Few-Shot Learning: External Data and Fine-Tuning Make a Differencecited by 123
- 2020Adversarial Feature Hallucination Networks for Few-Shot Learningcited by 110
- 2019Infinite Mixture Prototypes for Few-Shot Learningcited by 99
- 2018Delta-encoder: an effective sample synthesis method for few-shot object recognitioncited by 89
- 2020PACOH: Bayes-Optimal Meta-Learning with PAC-Guaranteescited by 85
- 2017Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theorycited by 76
- 2020Self-training for Few-shot Transfer Across Extreme Task Differencescited by 75
- 2021Relational Embedding for Few-Shot Classificationcited by 74
- 2013A PAC-Bayesian bound for Lifelong Learningcited by 69
- 2019Image Deformation Meta-Networks for One-Shot Learningcited by 56
- 2021Rectifying the Shortcut Learning of Background for Few-Shot Learningcited by 51
- 2021Boosting the Generalization Capability in Cross-Domain Few-shot Learning via Noise-enhanced Supervised Autoencodercited by 51
- 2020Prototype Completion with Primitive Knowledge for Few-Shot Learningcited by 50
- 2022Joint Distribution Matters: Deep Brownian Distance Covariance for Few-Shot Classificationcited by 50
- 2020DeepEMD: Differentiable Earth Mover's Distance for Few-Shot Learningcited by 49
- 2021Dynamic Distillation Network for Cross-Domain Few-Shot Recognition with Unlabeled Datacited by 49
- 2021Meta-Learning with Task-Adaptive Loss Function for Few-Shot Learningcited by 49
- 2019A Theoretical Analysis of the Number of Shots in Few-Shot Learningcited by 39
- 2022Generating Representative Samples for Few-Shot Classificationcited by 39
- 2022Understanding Cross-Domain Few-Shot Learning Based on Domain Similarity and Few-Shot Difficultycited by 37
- 2022Rethinking Generalization in Few-Shot Classificationcited by 36
50 most cited of 68 All 68 (JSON)
Built on by
Citing papers per year: 2024: 1.
1 paper.