mlsanityData to 7 Oct 2026

Few-shot Learning with Global Relatedness Decoupled-Distillation

arXiv 2107.05583 ↗First posted 12 Jul 2021cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Despite the success that metric learning based approaches have achieved in few-shot learning, recent works reveal the ineffectiveness of their episodic training mode. In this paper, we point out two potential reasons for this problem: 1) the random episodic labels can only provide limited supervision information, while the relatedness information between the query and support samples is not fully exploited; 2) the meta-learner is usually constrained by the limited contextual information of the local episode. To overcome these problems, we propose a new Global Relatedness Decoupled-Distillation (GRDD) method using the global category knowledge and the Relatedness Decoupled-Distillation (RDD) strategy. Our GRDD learns new visual concepts quickly by imitating the habit of humans, i.e. learning from the deep knowledge distilled from the teacher. More specifically, we first train a global learner on the entire base subset using category labels as supervision to leverage the global context information of the categories. Then, the well-trained global learner is used to simulate the query-support relatedness in global dependencies. Finally, the distilled global query-support relatedness is explicitly used to train the meta-learner using the RDD strategy, with the goal of making the meta-learner more discriminative. The RDD strategy aims to decouple the dense query-support relatedness into the groups of sparse decoupled relatedness. Moreover, only the relatedness of a single support sample with other query samples is considered in each group. By distilling the sparse decoupled relatedness group by group, sharper relatedness can be effectively distilled to the meta-learner, thereby facilitating the learning of a discriminative meta-learner. We conduct extensive experiments on the miniImagenet and CIFAR-FS datasets, which show the state-of-the-art performance of our GRDD method.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 53 references that are in the map. First posted on arXiv 12 Jul 2021.

Builds on

42 of 53 references.

References · 42

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  3. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  4. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  6. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  7. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  8. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  9. 2016Matching Networks for One Shot Learningcited by 2,246
  10. 2017Learning to Compare: Relation Network for Few-Shot Learningcited by 1,139
  11. 2018Dual Attention Network for Scene Segmentationcited by 1,001
  12. 2014Neural Turing Machinescited by 630
30 more
  1. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
  2. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  3. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 435
  4. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  5. 2018Meta-learning with differentiable closed-form solverscited by 395
  6. 2018Dynamic Few-Shot Visual Learning without Forgettingcited by 385
  7. 2018Meta-Transfer Learning for Few-Shot Learningcited by 306
  8. 2019A Baseline for Few-Shot Image Classificationcited by 298
  9. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 293
  10. 2018Learning to Propagate Labels: Transductive Propagation Network for Few-shot Learningcited by 208
  11. 2018Generalized Zero- and Few-Shot Learning via Aligned Variational Autoencoderscited by 206
  12. 2018Low-Shot Learning from Imaginary Datacited by 181
  13. 2021Free Lunch for Few-shot Learning: Distribution Calibrationcited by 165
  14. 2018Task-Agnostic Meta-Learning for Few-shot Learningcited by 151
  15. 2017Few-Shot Image Recognition by Predicting Parameters from Activationscited by 122
  16. 2020Interventional Few-Shot Learningcited by 112
  17. 2019Prototype Rectification for Few-Shot Learningcited by 96
  18. 2020Laplacian Regularized Few-Shot Learningcited by 90
  19. 2018Delta-encoder: an effective sample synthesis method for few-shot object recognitioncited by 89
  20. 2020Boosting Few-Shot Learning With Adaptive Margin Losscited by 83
  21. 2019Transductive Episodic-Wise Adaptive Metric for Few-Shot Learningcited by 67
  22. 2017Learning to Remember Rare Eventscited by 63
  23. 2019Few-Shot Learning with Embedded Class Models and Shot-Free Meta Trainingcited by 57
  24. 2019Few-Shot Learning via Saliency-guided Hallucination of Samplescited by 49
  25. 2019Few-Shot Learning with Global Class Representationscited by 44
  26. 2017Low-shot learning with large-scale diffusioncited by 33
  27. 2019PARN: Position-Aware Relation Networks for Few-Shot Learningcited by 27
  28. 2019Adaptive Posterior Learning: few-shot learning with a surprise-based memory modulecited by 25
  29. 2020Cooperative Bi-path Metric for Few-shot Learningcited by 15
  30. 2020Memory-Augmented Relation Network for Few-Shot Learningcited by 10

Built on by

Not cited yet.