mlsanityData to 14 Oct 2026

Like Humans to Few-Shot Learning through Knowledge Permeation of Vision and Text

arXiv 2405.12543 ↗First posted 21 May 2024cs.CV cs.AIcited by 0

Not on the map. Why?

Abstract (arXiv)

Few-shot learning aims to generalize the recognizer from seen categories to an entirely novel scenario. With only a few support samples, several advanced methods initially introduce class names as prior knowledge for identifying novel classes. However, obstacles still impede achieving a comprehensive understanding of how to harness the mutual advantages of visual and textual knowledge. In this paper, we propose a coherent Bidirectional Knowledge Permeation strategy called BiKop, which is grounded in a human intuition: A class name description offers a general representation, whereas an image captures the specificity of individuals. BiKop primarily establishes a hierarchical joint general-specific representation through bidirectional knowledge permeation. On the other hand, considering the bias of joint representation towards the base set, we disentangle base-class-relevant semantics during training, thereby alleviating the suppression of potential novel-class-relevant information. Experiments on four challenging benchmarks demonstrate the remarkable superiority of BiKop. Our code will be publicly available.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 49 of its 68 references that are in the map. First posted on arXiv 21 May 2024.

Builds on

49 of 68 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,897
  6. 2017Prototypical Networks for Few-shot Learningcited by 3,015
  7. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  8. 2015VQA: Visual Question Answeringcited by 2,506
  9. 2021Learning to Prompt for Vision-Language Modelscited by 2,281
  10. 2016Matching Networks for One Shot Learningcited by 2,255
  11. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  12. 2022Conditional Prompt Learning for Vision-Language Modelscited by 1,433
37 more
  1. 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,423
  2. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  3. 2018On First-Order Meta-Learning Algorithmscited by 1,018
  4. 2018Meta-Learning with Latent Embedding Optimizationcited by 476
  5. 2019Meta-Learning with Differentiable Convex Optimizationcited by 461
  6. 2018Meta-Learning for Semi-Supervised Few-Shot Classificationcited by 456
  7. 2020Rethinking Few-Shot Image Classification: a Good Embedding Is All You Need?cited by 437
  8. 2018TADAM: Task dependent adaptive metric for improved few-shot learningcited by 408
  9. 2018Meta-learning with differentiable closed-form solverscited by 397
  10. 2018Few-Shot Learning via Embedding Adaptation with Set-to-Set Functionscited by 294
  11. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  12. 2020Meta-Baseline: Exploring Simple Meta-Learning for Few-Shot Learningcited by 179
  13. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  14. 2020CrossTransformers: spatially-aware few-shot transfercited by 174
  15. 2019Boosting Few-Shot Visual Learning with Self-Supervisioncited by 140
  16. 2022Causality Inspired Representation Learning for Domain Generalizationcited by 138
  17. 2021Visformer: The Vision-friendly Transformercited by 137
  18. 2019Charting the Right Manifold: Manifold Mixup for Few-shot Learningcited by 136
  19. 2019Adaptive Cross-Modal Few-Shot Learningcited by 95
  20. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  21. 2020Boosting Few-Shot Learning With Adaptive Margin Losscited by 83
  22. 2019Associative Alignment for Few-shot Image Classificationcited by 58
  23. 2021Subspace Regularizers for Few-Shot Class Incremental Learningcited by 56
  24. 2021Shallow Bayesian Meta Learning for Real-World Few-Shot Recognitioncited by 48
  25. 2022Rethinking Generalization in Few-Shot Classificationcited by 36
  26. 2022Few-Shot Classification with Contrastive Learningcited by 31
  27. 2023Semantic Prompt for Few-Shot Image Recognitioncited by 29
  28. 2021Binocular Mutual Learning for Improving Few-shot Classificationcited by 26
  29. 2022Matching Feature Sets for Few-Shot Image Classificationcited by 25
  30. 2020Mixture-based Feature Space Learning for Few-shot Image Classificationcited by 23
  31. 2022Self-Promoted Supervision for Few-Shot Transformercited by 22
  32. 2017Gaussian Prototypical Networks for Few-Shot Learning on Omniglotcited by 20
  33. 2022Channel Importance Matters in Few-Shot Image Classificationcited by 20
  34. 2022Self-Supervision Can Be a Good Few-Shot Learnercited by 19
  35. 2021Aligning Visual Prototypes with BERT Embeddings for Few-Shot Learningcited by 12
  36. 2021Learning Instance and Task-Aware Dynamic Kernels for Few Shot Learningcited by 4
  37. 2023Contrastive Meta-Learning for Partially Observable Few-Shot Learningcited by 2

Built on by

Not cited yet.