mlsanityData to 14 Oct 2026

Multimodal Prototypical Networks for Few-shot Learning

arXiv 2011.08899 ↗First posted 17 Nov 2020cs.CV cs.LGcited by 48

Not on the map. Why?

Abstract (arXiv)

Although providing exceptional results for many computer vision tasks, state-of-the-art deep learning algorithms catastrophically struggle in low data scenarios. However, if data in additional modalities exist (e.g. text) this can compensate for the lack of data and improve the classification results. To overcome this data scarcity, we design a cross-modal feature generation framework capable of enriching the low populated embedding space in few-shot scenarios, leveraging data from the auxiliary modality. Specifically, we train a generative model that maps text data into the visual feature space to obtain more reliable prototypes. This allows to exploit data from additional modalities (e.g. text) during training while the ultimate task at test time remains classification with exclusively visual data. We show that in such cases nearest neighbor classification is a viable approach and outperform state-of-the-art single-modal and multimodal few-shot learning methods on the CUB-200 and Oxford-102 datasets.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). First posted on arXiv 17 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 13; 2022: 8; 2023: 18; 2024: 4; 2025: 3; 2026: 2.

48 papers.

Sort:

By area:

  1. 2022A Comprehensive Survey of Few-shot Learning: Evolution, Applications, Challenges, and Opportunitiescited by 163
  2. 2023Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Modelscited by 91
  3. 2021Hyperbolic Busemann Learning with Ideal Prototypescited by 41
  4. 2023CLIP-guided Prototype Modulating for Few-shot Action Recognitioncited by 41
  5. 2021Multimodal Co-learning: Challenges, Applications with Datasets, Recent Advances and Future Directionscited by 35
  6. 2022SgVA-CLIP: Semantic-guided Visual Adapting of Vision-Language Models for Few-shot Image Classificationcited by 34
  7. 2023Meta-learning approaches for few-shot learning: A survey of recent advancescited by 25
  8. 2021Interactive Disentanglement: Learning Concepts by Interacting with their Prototype Representationscited by 24
  9. 2021Imbalanced Big Data Oversampling: Taxonomy, Algorithms, Software, Guidelines and Future Directionscited by 21
  10. 2021A Closer Look at Prototype Classifier for Few-shot Image Classificationcited by 15
  11. 2022Multimodal Prototype-Enhanced Network for Few-Shot Action Recognitioncited by 13
  12. 2021Multimodality in Meta-Learning: A Comprehensive Surveycited by 11
36 more
  1. 2023Towards Effective Visual Representations for Partial-Label Learningcited by 11
  2. 2021HetMAML: Task-Heterogeneous Model-Agnostic Meta-Learning for Few-Shot Learning Across Modalitiescited by 8
  3. 2023Episodic Multi-Task Learning with Heterogeneous Neural Processescited by 8
  4. 2023Visual Hindsight Self-Imitation Learning for Interactive Navigationcited by 6
  5. 2023Hierarchical Explanations for Video Action Recognitioncited by 5
  6. 2023The Multimodal And Modular Ai Chef: Complex Recipe Generation From Imagerycited by 5
  7. 2023ProtoEM: A Prototype-Enhanced Matching Framework for Event Relation Extractioncited by 5
  8. 2021Dizygotic Conditional Variational AutoEncoder for Multi-Modal and Partial Modality Absent Few-Shot Learningcited by 4
  9. 2022Kernel Relative-prototype Spectral Filtering for Few-shot Learningcited by 4
  10. 2023Consistency Prototype Module and Motion Compensation for Few-Shot Action Recognition (CLIP-CPM2C)cited by 4
  11. 2024DMSD-CDFSAR: Distillation from Mixed-Source Domain for Cross-Domain Few-shot Action Recognitioncited by 4
  12. 2021Few-shot Partial Multi-view Learningcited by 3
  13. 2021Predicting emergent linguistic compositions through time: Syntactic frame extension via multimodal chainingcited by 3
  14. 2023Robust Text Classification: Analyzing Prototype-Based Networkscited by 3
  15. 2021Shaping Visual Representations with Attributes for Few-Shot Recognitioncited by 2
  16. 2023Task-Specific Alignment and Multiple Level Transformer for Few-Shot Action Recognitioncited by 2
  17. 2023On Task-personalized Multimodal Few-shot Learning for Visually-rich Document Entity Retrievalcited by 2
  18. 2022Prototype-Aware Heterogeneous Task for Point Cloud Completioncited by 1
  19. 2022Text2Model: Text-based Model Induction for Zero-shot Image Classificationcited by 1
  20. 2022Disentangled Generation with Information Bottleneck for Few-Shot Learningcited by 1
  21. 2023LPN: Language-guided Prototypical Network for few-shot classificationcited by 1
  22. 2024Controllable Relation Disentanglement for Few-Shot Class-Incremental Learningcited by 1
  23. 2025Explainable AI the Latest Advancements and New Trendscited by 1
  24. 2021Will Multi-modal Data Improves Few-shot Learning?cited by 0
  25. 2021Multimodal Classification: Current Landscape, Taxonomy and Future Directionscited by 0
  26. 2022Semantic Cross Attention for Few-shot Learningcited by 0
  27. 2023A Systematic Evaluation and Benchmark for Embedding-Aware Generative Models: Features, Models, and Any-shot Scenarioscited by 0
  28. 2023SENet: A Spectral Filtering Approach to Represent Exemplars for Few-shot Learningcited by 0
  29. 2023HAVE-Net: Hallucinated Audio-Visual Embeddings for Few-Shot Classification with Unimodal Cuescited by 0
  30. 2023Heterogenous Memory Augmented Neural Networkscited by 0
  31. 2024Episodic-free Task Selection for Few-shot Learningcited by 0
  32. 2024Automated Privacy-Preserving Techniques via Meta-Learningcited by 0
  33. 2025Doodle Your Keypoints: Sketch-Based Few-Shot Keypoint Detectioncited by 0
  34. 2025ProtoTopic: Prototypical Network for Few-Shot Medical Topic Modelingcited by 0
  35. 2026IPEC: Test-Time Incremental Prototype Enhancement Classifier for Few-Shot Learningcited by 0
  36. 2026Scalable Semantic Steering of Embedding Projectionscited by 0